I'm preparing to work on an application that should support different countries, so I need support for:
currencies
numbers formatting (for prices etc.)
Switching locales will be based on http://jmsyst.com/bundles/JMSI18nRoutingBundle.
Now I need to store information about locale specific stuff like currency, currency divisor, precision etc + some other country specific data.
What is the best way of doing that in Symfony 2? I know there's a Money field type in SF2 forms, but I will need to display prices etc all over the app, so maybe there's a bundle or something that might help?
I don't want to store information like that in config "parameters", rather in a DB table maybe and use a number_format then, but maybe someone has better ideas?
I'd really appreciate any input here! :)
Edit
One more thing came to my mind, validation constraints may differ for each country, for example phone number might have different lenghts in DE and, for example, in UK. Is it possible to define constraints based on locale? Or do I have to create my own constraint and inside that check locale etc?
I stumbled over this bundle recently, might be what you're looking for:
https://github.com/craue/TwigExtensionsBundle
Related
We want to allow our web users to define a timetable for themselves. However because the events are very varied the information may not be exactly the same so the column headings and number of columns vary. (We may be forced to define it in general terms - so we define the column headings and they enter the data.) They also need to be able to sort and filter. Import from spreadsheet would also be good as would printing or exporting.
We are using PHP & Laravel Framework.
Does anyone know of something that might help us do this quickly and efficiently?
Thank you
My table is (device, type, value, timestamp), where (device,type,timestamp) makes a unique combination ( a candidate for composite key in non-DynamoDB DBMS).
My queries can range between any of these three attributes, such as
GET (value)s from (device) with (type) having (timestamp) greater than <some-timestamp>
I'm using dynamoosejs/dynamoose. And from most of the searches, I believe I'm supposed to use a combination of the three fields (as a single field ; device-type-timestamp) as id. However the set: function of Schema doesn't let me use the object properties (such as this.device) and due to some reasons, I cannot do it externally.
The closest I got (id:uuidv4:hashKey, device:string:GlobalSecIndex, type:string:LocalSecIndex, timestamp:Date:LocalSecIndex)
and
(id:uuidv4:rangeKey, device:string:hashKey, type:string:LocalSecIndex, timestamp:Date:LocalSecIndex)
and so on..
However, while using a Query, it becomes difficult to fetch results of particular device,type as the id, (hashKey or rangeKey) keeps missing from the scene.
So the question. How would you do it for such kind of table?
And point to be noted, this table is meant to gather content from IoT devices, which is generated every 5 mins by each device on an average.
I'm curious why you are choosing DynamoDB for this task. Advanced queries like this seem to be much better suited for a SQL based database as opposed to a NoSQL database. Due to the advanced nature of SQL queries, this task in my experience is a lot easier in SQL databases. So I would encourage you to think about if DynamoDB is truly the right system for what you are trying to do here.
If you determine it is, you might have to restructure your data a little bit. You could do something like having a property that is device-type and that will be the device and type values combined. Then set that as an index, and query based on that and sort by the timestamp, and filter out the results that are not greater than the value you want.
You are correct that currently, Dynamoose does not pass in the entire object into the set function. This is something that personally I'm open to exploring. I'm a member on the GitHub project, and if you would like to submit a PR adding that feature I would be more than happy to help explore that option with you and get that into the codebase.
The other thing you might want to explore is having a DynamoDB stream, that will set that device-type property whenever it gets added to your DynamoDB table. That would abstract that logic out of DynamoDB and your application. I'm not sure if it's necessary for what you are doing to decouple it to that level, but it might be something you want to explore.
Finally, depending on your setup, you could figure out which item will be more unique, device or type, and setup an index on that property. Then just query based on that, and filter out the results of the other property that you don't want. I'm not sure if that is what you are looking for, it will of course work, but I'm not sure how many items you will have in your table, and there get to be questions about scalability at a certain level. One way to solve some of those scalability questions might be to set the TTL of your items if you know that you the timestamp you are querying for is constant, or predictable ahead of time.
Overall there are a lot of ways to achieve what you are looking to do. Without more detail about how many items, what exactly those properties will be doing, the amount of scalability you require, which of those properties will be most unique, etc. it's hard to give a good solution. I would highly encourage you to think about if NoSQL is truly the best way to go. That query you are looking to do seems a LOT more like a SQL query. Not saying it's impossible in DynamoDB, but it will require some thought about how you want to structure your data model, and such.
Considering opinion of #charlie-fish, I decided to jump into Dynamoose and improvise the code to pass the model to the set function of the attribute. However, I discovered that the model is already being passed to default parameter of the attribute. So I changed my Schema to the following:
id:hashKey;default: function(model){ return model.device + "" + model.type; }
timestamp:rangeKey
For anyone landing here on this answer, please note that the default & set functions can access attribute options & schema instance using this . However both those functions should be regular functions, rather than arrow functions.
Keeping this here as an answer, but I won't accept it as an answer to my question for sometime, as I want to wait for someone else to hit out a better approach.
I also want to make sure that if a value is passed for id field, it shouldn't be set. For this I can use set to ignore the actual incoming value, which I don't know how, as of yet.
I'm trying to find freebase co-types, that is, given a type you find 'compatible' types:
Suppose to start with /people/person, it might be a musician (/music/group_member), but not a music album (/music/album), i don't know if in freebase is there something like owl 'disjointWith' between types, anyway in the MQL cookbook they suggest to use this trick.
The query in the example get all instances of a given type, then get all types of all instances and let them to be unique...this is clever, but the query goes in timeout... is there another way? for me it's ok even a static list/result, i don't need the live query...i think the result would be the same...
EDIT:
Incompatible types type seems to be useful and similar to disjointWith, may also be used with the suggest...
Thanks!
luca
Freebase doesn't have the concept of disjointWith at the graph or schema level. The Incompatible Types base that you found is a manually curated version of that which may be used in a future version of the UI, but isn't today.
If you want to find all co-types, as they exist in the graph today, you can do that using the query you mentioned, but you're probably better off using the data dumps. I'd also consider establishing a frequency threshold to eliminate low frequency co-types so that you filter out mistakes and other noise.
As Tom mentioned, there are some thresholds that you might want to consider to filter out some of the less notable or experimental types that users have created in Freebase.
To give you an idea of what the data looks like I've run the query for all co-types for /people/person.
/common/topic (100.00%)
/book/author (23.88%)
/people/deceased_person (18.68%)
/sports/pro_athlete (12.49%)
/film/actor (9.72%)
/music/artist (7.60%)
/music/group_member (4.98%)
/soccer/football_player (4.53%)
/government/politician (3.92%)
/olympics/olympic_athlete (2.91%)
See the full list here.
You can also experiment with Freebase Co-Types using this app that I built (although it is prone to the same timeouts that you experienced).
I'm looking at accepting a project that would require me to clean up an existing e-commerce website. Its been relatively successful and has over 100,000 individual products - loaded both by the client and its publishers.
The site wasn't originally designed for this many products and has become fairly disorganized.
SO, the client has asked I look at a more robust search option - filterable and so forth. I completely agree it needs to be improved, but after looking at the database, I can tell that there are dozens and dozens of categories and not everything is labeled correctly etc.
Is there any database management software that could help me clean up 100,000 entries quickly? Make categories consistent - fix uppercase/lowercase problems etc.
Are there any companies out there that I can source just this particular part of the project to?
Its a massive amount of data-entry. If I spent 2 minutes per product, it would take me 6 months full time to just to complete the database cleanup. I either need to get it down to a matter of seconds per product or find a company that specializes in this type of work.
I don't even know what to search for on Google.
Thanks guys!
--
Thanks everyone for your ideas! I have a lot of options now so I feel a lot more comfortable heading in to this project. Right now I think the direction we will go is to build a tool that allows the client to hire data entry people that can update it as necessary. Then I will work as a consultant, taking care of any UPDATE-WHERE type functions as necessary.
Thanks again!
If there are inconsistencies like you are describing, it sounds like the problem may be more an issues of a bad data model (i.e. lack of normalization) than just dirty data. If good normalization is in place, cleaning up categories should be as simple as updating a single record per each category - but if category name is used instead of a foreign key, then you will most likely need to perform a series of UPDATE WHERE statements to clean up the text.
You may want to look into an ETL (extract, transform, load) tool that can help with bulk data transformation. I'm not familiar with ETL tools for mysql, but I'm sure they exist. SQL Server has a build in service called SQL Integration Services that provides the ability to extract data from an existing data source, perform bulk changes or transformations, and then reload the data back into a destination database. Tools like this may help speed up the process of standardizing capitalization, punctuation, changing categories etc.
Even still, don't overlook the possibility that the data model may need tweaking to help prevent this type of situation in the future.
Edit: Wikipedia has a list of opensource ETL products that you may want to investigate.
In any case you'll probability need to do more than "clean the data", which means you'll need to build new normalized tables. So start there, build a new database that is fully normalized, import the data "as is", with all the duplicate categories, etc.
for example, new tables:
Items
ItemID int identity/auto number
ItemName string
CategoryID int
....
Categories
CategoryID int identity/auto number
CategoryName string
....
import the bad data into the new system:
Items
ItemID ItemName CategoryID
1 thing A 1
2 thing B 2
3 thing C 3
4 thing D 1
Categories
CategoryID CategoryName
1 Game
2 food
3 games
now, you can consolidate the data using the PKs
UPDATE Items
SET CategoryID=1
WHERE CategoryID=3
DELETE Categories
WHERE CategoryID=3
You might just write an application where the customer can do the consolidation. Let them select the duplicates on a screen and merge to a selected parent category. you have this application do the merge sql from above.
If there are issues of needing to have a clean cut over date, create an application that generates a series of "Map" tables, where you store the CategoryNameOld="games" and the CategoryNameNew="Game" and use these when you do the conversion/load of the bad data into the new system's tables.
I would implement the new search system or whatever and build them a tool that would allow them to easily go through and cleanup the listings, re-categorize, etc. This task requires domain knowledge, so they're the best ones to do it.
Do some number crunching so they can prioritize the list and clean in order of importance.
Keep in mind that one or your options is to build a crappy interface that somebody can use to edit records, hire half a dozen data-entry people from a temp agency, spend two days training them, and let them go to town.
Is storing a list of 1000 instance of my custom class to the session variable a good approach ? My asp.net web app need multilingual support and i am storing the labels in a table.I will have multiple users who has their own language preference and text(label content) preference.
I am thinking about loading the labels and store it in session variable and access that to use in the pages.Storing it in Application variable does not work because each customer have their own text for the labels.So is storing session a good way of doing this ? I guess i will have almost 1000 labels as of now and it may increase as the application grows.
My custom class has 2 proprties. LanguageCode and LanguageName
for some reason i cant use asp.net Resource files :(
Whats your thoughts on this ?
You should store a single set of labels for each language, then store the language name in session and use it to look up the correct labelset.
Some thoughts to notice:
If your managers have something in particular against resx's, you can store all those labels in any other format (e.g. plain text files), or in a DB.
If you have little amount of users, and loading time is extremely crucial, your managers may have a point. Other than that, they're wrong, and I would consider trying to explain that to them.
Try considering "computing" the labels at runtime (e.g. if some of them include adding prefixes, suffixes, etc. you can save only the "stems" and provide a relevant label only on demand. That will save you some server space).
If I were you I'd first spend some time trying to figure out why your resource files are not working... as long as you are setting the Thread.CurrentThread.CurrentCulture (or is it CurrentUICulture?) value to the specific culture, and have the resource files in the correct place, I can't think of any reasons as to why it wouldn't work.
Not a good idea, there are two problems with it:
It will use up the memory on the server, and reduce the number of users that can use the system
The time for each request will increase
The standard way to do this is using RESX (resource) files. You should try to fix that.
Why not use Application variable and create as many Application variables as the languages you have. Each application variable will be a superset of all the labels for that language.
Or maybe store the entire table(s) in application variable(s), instead of storing multiple session variables that, I assume, they intersect with each other.