New to DynamoDB over here. I need to figure out a DynamoDB schema for different locations inside a building. Additionally, I need to be able to identify computers assigned to each of the locations. The locations are nested inside other locations. For instance,
Building 1
Wing A
Floor 1
Section A
Office 1
Computer A
Computer B
Office 2
Computer A
Section B
Office 1
... and so on.
ACCESS PATTERNS:
Show all of the locations (wings, Floors, sections, etc) in a building
Show a specific location
Show all of the computers assigned to a specific location
Show the location of a specific computer
What I was thinking:
I initially wanted to create something like this:
PartitionKey SortKey Attributes
Building#1 Building#1 (For metadata)
Building#1 Section#1 [...]
Building#1 Section#1|Section#2 [...]
Building#1 Section#1|Section#2|Section#3 [...]
I know this is the wrong way to think about it, but I can't figure out any other way.
What is the best way to model the location of sections, offices, etc of a building?
If those are really the only access patterns you can probably do something with a simple GSI. I wouldn't using Building as the PartitionKey because this will give you a lot of hot spots in the data. Something like this would probably work:
PartitionKey SortKey GSI_PartitionKey GSI_SortKey Attributes
Building#1 'Location' [...]
Wing#1 'Location' 'Location' Building#1 . [...]
Floor#1 'Location' 'Location' Building#1|Wing#A [...]
.
.
.
Computer#1 'Computer' 'Computer' B#1|W#A|F#1|S#A|O#1 [...]
Computer#2 'Computer' 'Computer' B#1|W#A|F#1|S#B|O#1 [...]
.
.
.
The SortKey values here are more optional, but they tend to allow for changes later without as much work now.
To get all the locations in a building you query the GSI where the GSI_PartitionKey is 'Location' and the GSI_SortKey begins with your building ID. You can add sub locations to the string so you can get all the locations in Wing A with a begins with of Building#1|Wing#A|
Get a specific location using the PartitionKey (and optionally the SortKey = 'Location').
To get all the computers in a locations GSI where the GSI_PartitionKey is 'Computer' and the GSI_SortKey begins with your location ID.
Get a specific computer using the PartitionKey (and optionally the SortKey = 'Computer') the attributes should include it's location.
I think you're on the right track...
Having the hierarchical data coded as a delimited sort key seems to follow the recommendations I've seen (though your two sets of example data don't match) Section#1|Section#2|Section#3 vs Wing A|Floor 1|Section A
I'd probably consider having the table with just a hash of "serial number" or "asset ID"
Then have a GSI with the key's you describe.
Related
We often use GraphBLAS for graph processing so we need to use the incidence matrix. I haven't been able to find a way to export this from Grakn to a csv or any file. Is this possible?
There isn't a built-in way to dump data to CSV in Grakn right now. However, we do highly encourage our community to contribute open source tooling for these kinds of tasks! Feel free to chat to use about it on our discord.
As to how it can be done, conceptually it's pretty easy:
Query to get stream all hyper-relations out:
match $r isa relation;
and then for each relation, we can pipeline another query (possibly in new transaction if you wish to keep memory usage lower):
match $r iid <iid of $r from previous query>; $r ($x); get $x;
which will get you everything in this particular hyper relation $r playing a role.
If you also wish to extract attributes that are attached to the hyper relation, you can use the following
match $r iid <iid of $r from first query>; $r has $a; get $a;
In effect we can use these steps to build up each column in the A incidence matrix.
There are a couple if important caveats I should bring up:
What you'll end up with, will exclude all type information about the hyper relations, the role players in the relations, and the actual role that is being played by the role player, and attribute types owned.
==> It would be interesting to hear/discuss how one could encode types information for use in GraphBLAS
In Graql, it's entirely possible to have relations participating in relations. in the worst case, this means all hyper-edges E will also be present in the set V. In practice only a few relations will play a role in other relations, so only a subset of E may be in V.
So the incidence matrix is equivalent to the nodes/edges array used in force graph visualisation. In this case it is pretty straight forward.
My approach would be slightly different than the above as all i need to do is pull all of the things in the db (entities, relations, attributes), with
match $ting isa thing;
Now when i get my transaction back, for each $ting I want to pull all of the available properties using both local and remote methods if I am building a force graph viz, but for your incidence matrix, I really only care about pulling 3 bits of data:
The iid of the thing
The attributes the thing may own.
The roles the thing owns if it is a relation
Essentially one tests each returned object to find out the type (e.g. entity, attribute, relation), and then uses some of the local and remote methods to get the data one wants. In Python, the code for pulling the data for relations looks like
# pull relation data
elif thing.is_relation():
rel = {}
rel['type'] = 'relation'
rel['symbol'] = key
rel['G_id'] = thing.get_iid()
rel['G_name'] = thing.get_type().get_label().name()
att_obj = thing.as_remote(r_tx).get_has()
att = []
for a in att_obj:
att.append(a.get_iid())
rel['has'] = att
links = thing.as_remote(r_tx).get_players_by_role_type()
logger.debug(f' links are -> {links}')
edges = {}
for edge_key, edge_thing in links.items():
logger.debug(f' edge key is -> {edge_key}')
logger.debug(f' edge_thing is -> {list(edge_thing)}')
edges[edge_key.get_label().name()] = [e.get_iid() for e in list(edge_thing)]
rel['edges'] = edges
res.append(rel)
layer.append(rel)
logger.debug(f'rel -> {rel}')
This then gives us a node array, which we can easily process to build an edges array (i.e. the links joining an object and the attributes it owns, or the links joining a relation to its role players). Thus, exporting your incidence matrix is pretty straightforward
I'm trying to get list of Chinese universities and their adresses. The minimum being the City/Town name. I will use these addresses to populate a googlemap, fiddle here.
I saw interesting code such as:
SELECT ?resource ?value
WHERE {
?resource a <http://dbpedia.org/class/yago/CitiesAndTownsInDenmark> .
?resource <http://dbpedia.org/property/populationTotal> ?value .
FILTER (?value > 100000)
}
ORDER BY ?resource ?value
Since CitiesAndTownsInChina doesn't work,
1. Where to find the exact name of the class I'am targeting ? and
2. Where to find dbpedia's operators manual ?
Note: I'am a very active user on Wikipedia, I'am well aware of all the data available there, but the dbpedia ontology/syntaxe/keywords is quite hard to get.
Personal note: queries on http://dbpedia.org/snorql/ , http://dbpedia.org/sparql/ , http://querybuilder.dbpedia.org/
(Expanding on my reply to How to find cities with more than X population in a certain country)
CitiesAndTownsInDenmark exists because people use the category http://en.wikipedia.org/wiki/Category:Cities_and_towns_in_Denmark in wikipedia. Wikipedia categories are pretty loose and as a result there's a lot of variation in style, so even if a useful category exists the name may not be guessable.
In addition categories are maintained manually, and may not be consistently applied.
A good place to start is looking at the data. Visiting http://dbpedia.org/page/Beijing I see yago:MetropolitanAreasOfChina which seems promising, but if you follow that link you'll see it's not well populated.
As a consequence avoid relying on the existence of such categories and directly querying for populated places in a country. This information comes from wikipedia infoboxes, and they're much more consistent than categories. Taking Beijing as an exemplar again I found:
select ?s {
?s a <http://dbpedia.org/ontology/PopulatedPlace> ;
<http://dbpedia.org/ontology/country> <http://dbpedia.org/resource/China>
}
(The relevant properties and values for my query were found by copying link location in the Beijing page)
with the result:
"http://dbpedia.org/resource/Hulunbuir"
"http://dbpedia.org/resource/Guangzhou"
"http://dbpedia.org/resource/Chongqing"
"http://dbpedia.org/resource/Kuqa_County"
"http://dbpedia.org/resource/Changzhou"
... nearly 3000 results ...
You'll notice that position is encoded multiple times (geo:lat and long, georss:point, various dbpprop:latd longd things), and there seem to be two values excitingly. You can either simply deal with the multiple values in whichever format you prefer, or try picking just one using GROUP BY and SAMPLE.
As for a manual, almost everything I know of are academic papers, and not very useful. However the data is reasonably self documenting.
for your first question:
you can see possible classes by querying one member of your intended set of entities (ex: Shanghai).
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
SELECT ?type WHERE {
<http://dbpedia.org/resource/Shanghai> rdf:type ?type.
FILTER regex(str(?type), ".*China", "i").
} LIMIT 100
which gives this result:
dbpedia:class/yago/MetropolitanAreasOfChina [http]
dbpedia:class/yago/PortCitiesAndTownsInChina [http]
dbpedia:class/yago/MunicipalitiesOfThePeople'sRepuBlicOfChina [http]
dbpedia:class/yago/PopulatedCoastalPlacesInChina [http]
they are CamelCase versions of the categories that you will find at the bottom of wikipedia pages. I was fooled for a while by the erroneous capitalization of RepuBlic and finally saw that it contains only 4 cities, so it is of limited use for you.
so I would propose to go with #user205512 answer and get the cities by linking 2 properties.
for your second question:
I would advice you to search/ask on http://answers.semanticweb.com
As I explained in my post a few days before I'm programming an ASP MVC3 multi-language website which should contain facetted search, full text search and a distance search. To realize that I've installed solr 3.3 on a Tomcat 7. I'm also successfully integrated a dataimporthandler.
Now I want to index the data from my relational ms sql database. I read the index structure looks like one table containing all the data of one object. That means if I've got a object like a car my schema catains fields like Branding, Color and so on.
But what about n-m realtions? Does the index "table" have one column for each relation?
And what about multi language items? Should I create one object/row int the index for each language?
And should I save just the id of objects in the index or the whole names?
And last how to index (query) a Object like on the database image? (I read something about dynamic fields and multiplevalue fields but I'm not sure if it is the solution for my problem)
I've a example of a database design I'm talking about attached.
Thanks for all the answers!!!
Update:
The people should be able to have different way to search.
They should have the possibility to search the tbl_text_local.text by full text searching and the miscellaneous are are facettes.
The Result should be a list of objects that match to the search and a list of facetts.
But how should I index the Miscellaneous? Is there a posibility to index them in a form like that:
<cattegory name = "cat1">
<Miscellaneous>
name...
</Miscellaneous>
<Miscellaneous>
...
</Miscellaneous>
<Miscellaneous>
...
</Miscellaneous>
</cattegory>
<cattegory name = "cat2">
<Miscellaneous>
</Miscellaneous>
<Miscellaneous>
</Miscellaneous>
<Miscellaneous>
</Miscellaneous>
</cattegory>
People should have a searchfield like:
Text input (to search the text)
Facettes:
Miscellaneous-Cattegory1
Miscellaneous1 (9)
Miscellaneous2 (39)
Miscellaneous3 (49)
Miscellaneous-Cattegory2
Miscellaneous5 (59)
Miscellaneous6 (69)
Miscellaneous-Cattegory3
Miscellaneous7 (7)
Miscellaneous8 (8)
Miscellaneous-Cattegory4
Miscellaneous9 (19)
There is no single, "best" way to model relationships in Solr. Unlike relational databases, where you design tables by following normalization, in Solr the schema design is very much ad-hoc, a function of the searches you will perform on the index. Ask yourself these questions as guidance:
What are users searching for? What is the "result type"? The schema should be designed around this.
What information do I need to facet?
What information do I need to include in full-text search?
What information do I need to use to sort results?
What information will I search by? I.e. what information will I use to filter search results, and how will I use that information?
What will I process at index-time and what will I process at query-time?
Finally, don't be afraid of duplicating data in the index for specific search purposes.
We have a db driven asp.net /sql server website and would like to investigate how we can allow users to create a new database category and fields - is this crazy?. Is there any examples of such organic websites out there - the fact that I havent seen any maybe suggest i am?
Interested in the best approach which would allow some level of control by Admin.
I've implemented things along these lines with a dictionary table, rather than a more traditional table.
The dictionary table might look something like this:
create table tblDictionary
(id uniqueidentifier, --Surrogate Key (PK)
itemid uniqueidentifier, --Think PK in a traditional database
colmn uniqueidentifier, --Think "column name" in a traditional database
value nvarchar, --Can hold either string or number
sortby integer) --Sorting columns may or may not be needed.
So, then, what would have been one row in a traditional table would become multiple rows:
Traditional Way (of course I'm not making up GUIDs):
ID Type Make Model Year Color
1 Car Ford Festiva 2010 Lime
...would become multiple rows in the dictionary:
ID ITEMID COLUMN VALUE
0 1 Type Car
1 1 CarMake Ford
2 1 CarModel Festiva
3 1 CarYear 2010
4 1 CarColor Lime
Your GUI can search for all records where itemid=1 and get all of the columns it needs.
Or it can search for all records where itemid in (select itemid from tblDictionary where column='Type' and value='Car' to get all columns for all cars.
In theory, you can put the user-defined types into the same table (Type='Type') as well as the user-defined columns that that Type has (Type='Column', Column='ColumnName'). This is where the sortby column comes into it - to help build the the GUI in the correct order, if you don't want to rely on something else.
A number of times, though, I have felt that storing the user-defined dictionary elements in the dictionary was a bit too much drinking-the-kool-aid. Those can be separate tables because you already know what structure they need at design time. :)
This method will never have the speed or quality of reporting that a traditional table would have. Those generally require the developer to have pre-knowledge of the structures. But if the requirement is flexibility, this can do the job.
Often enough, what starts out as a user-defined area of my sites has had a later project to normalize the data for reporting, etc. But this allows users to get started in a limited way and work out their requirements before engaging the developers.
After all that, I just want to mention a few more options which may or may not work for you:
If you have SharePoint, users already have the ability to create
their own lists in this way.
Excel documents in a shared folder that are saved in such a way
to allow multiple simultaneous edits would also serve the purpose.
Excel documents, stored on the webserver and accessed via ODBC
would also serve as single-table databases like this.
I know this sounds like a very stupid question, but I've been trying to figure this out and I can't find anything on this, though it seems obvious that this should be doable.
I'm developing something that queries an OWL file through its API using the Manchester OWL Syntax and a HermiT reasoner. I alternatively also run queries on the DL Query tab in Protege 4 to check my query results on that file. The file is basically the Friend Of A Friend (FOAF) ontology with added data instances.
The problem: I can't find a way to query an instance by its unique identifier.
It's supposed to be the URI so e.g. for a Person object with URI http://xmlns.com/foaf/0.1/Andrew_Kuchling, I tried to run the following queries:
Person and URI value "http://xmlns.com/foaf/0.1/Andrew_Kuchling"
Person value "http://xmlns.com/foaf/0.1/Andrew_Kuchling"
Person and URI value "Andrew_Kuchling"
Person that "Andrew_Kuchling"
none of these work. (The URI is constructed by prefixing http://xmlns.com/foaf/0.1/ to whatever string you enter and URI is not actually identified in the ontology as a property).
In FOAF, mbox_sha1sum is the sha1sum of a mailbox. Mailboxes are uniquely connected to individuals. So I tried the following query:
Person and mbox_sha1sum value "mbox_sha1sum-property-value-here"
However, it doesn't even execute this query because it feels the query is incorrect. The mbox_sha1sum value is the same that I've added for this Person. It exists in Individuals.
It does execute this query:
Person and firstName value "Andrew"
All data properties are not specialised data types. Treated as literals by default.
I really don't know what I'm doing wrong. Can someone please please help?
I know this question has been posted more than a year ago, but I recently asked me the same question and found the following solution:
Just put the name of the individual/member you want to query for in curly brackets:
{MyIndividualEntityName}
This way you can even query for more than one individual at a time:
{IndividualName1,IndividualName2}
Please note that these queries form class expressions of so called enumarated classes. This means, that the result of this query will not simply be a list of all super or sub classes of all the individuals you have named in your query. Instead it refers to an anonymous class that consists of exactly the individuals you have listed.
An anonymous class is a class that isn't available from outside of your ontology, because there is no referrable owl entity for this class (no IRI to refer to).
You can find more information on the OWL Manchester Syntax in the offcial W3C specification.