I was reading on a website that:
the only readily available addresses for medium-size organizations are /24s which have the potentially negative impact of increasing the size of the global Internet's routing table.
and on another that:
If a single organisation has 16 class C: every internet backbone will need 16 entries, if they were of class B, they would be stored in one entry (??)
Can someone explain the tradeoff in giving out Class C?
Assuming that the 16 class C subnet:
222.204.1.0/24
222.204.2.0/24
222.204.3.0/24
...
222.204.16.0/24
the router table will look like:
222.204.1.0/24 to con1
222.204.2.0/24 to con1
222.204.3.0/24 to con1
...
222.204.16.0/24 to con1
So, there is 16 entries.
If use class B net, just need 1 subnet to contain such host, assuming it's 128.1.1.0/16.
we just need one entry:
128.1.1.0/16 to con1
But, considering VLSM, one entry maybe contains so much host's address.(if address is continuous)
Related
I have a table like this:
Transports
id (PK)
createDt
shipperId
carrierId
consigneeId
1
23
contact3
contact2
contact1
2
24
contact1
contact2
contact3
3
28
contact3
contact2
contact4
My access pattern is:
find all transports where a contact was either shipper, carrier or consignee sorted by createDt. E.g. entering contact1 should return records 1, 2.
How can I do this in DyanomoDB?
I thought about creating a GSI. But then I need to create a separate GSI for each column, which would mean I need to join the query results on the columns myself. Perhaps there is an easier way.
I'd create a GSI on the table and split your single record up into multiple ones.
That would make writes slightly more complex, because you write multiple entities, but I'd do something like this:
PK
SK
type
GSI1PK
GSI1SK
other attributes
TRANSP#1
TRANSP#1
transport
createDt, (shipperId, carrierId, consigneeId)...
TRANSP#1
CONTACT#SHIP
shipper-contact
CONTACT#contact3
TRANSP#1#SHIP
...
TRANSP#1
CONTACT#CARR
carrier-contact
CONTACT#contact2
TRANSP#1#CARR
...
TRANSP#1
CONTACT#CONS
consignee-contact
CONTACT#contact1
TRANSP#1#CONS
...
To get all information about a given Transport ID you do a query with PK=TRANSP#<id>
To get just the basic information about a given Transport, you can do a GetItem on PK=TRANSP#<id> and SK=TRANSP<id> (You could also duplicate the contact infos here if they're fairly static.)
To get all transports a contact is involved in, you do a PK=CONTACT#<id> and SK starts with TRANSP on GSI1
If you really need server-side sorting, you might choose a different GSI1SK, maybe prefix it with the dt value, but I'd probably just do that client side.
I am testing TitanDB + Cassandra now.
Graph Schema like this:
VERTEX: USER(userId), IP(ip), SESSION_ID(sessionId), DEVICE(deviceId)
EDGE: USER->IP, USER->SESSION_ID, USER->DEVICE
DATA SIZE: Vertex 100Million, Edge: 1 billion
Index: Vertex-Centric index on all kinds of edge . Index for userId, ip, sessionId, and deviceId.
Set Vertext partition for IP, DEVICE and SESSION_ID. Total 32 partition.
Cassandra hosts:AWS EC2 I2 (2xlage) x 24 .
Currently, every host hold about 30G data.
Usecase: give a userId with a edgeLabel, find out all related users by this edge's out vertex.
for example: g.V().has(T.label, 'USER').has('USER_ID', '12345').out('USER_IP').in().valueMap();
But this kinds of query is pretty slow, sometimes, hundreds seconds.
One user can have many related IP (hundreds), so from these IPs, it also can get lots of USERs (thousands).
Does Titan parallel query for this kind of query against all partition of backend storage??
I try to use limit:
g.V().has(T.label, 'USER').has('USER_ID', '12345').out('USER_IP').limit(50).in().limit(100).valueMap()
It's also slow. I hope this kinds of query can be done in 5seconds.
How the Titan limit() works? Get all result first, then 'limit' ??
How to increase the performance for it? Can anyone give some advice?
One quick perfomance gain you could get is from using Titan's Vertex Centric Indices this allows you to make very quick leaps from one vertex to another. For example you could try something like this:
mgmt = graph.openManagement()
userId = mgmt.getPropertyKey('userId')
userIp = mgmt.getEdgeLabel('USER_IP')
mgmt.buildEdgeIndex(userIp, 'userIdByUserIP', Direction.BOTH, Order.decr, time)
mgmt.commit()
To create a simple vertex centric index.
If you want to lookup multiple user ips from multiple user vertices then you could try using Titan-Hadoop. However, that is a more involved process.
So I add keys to my Redis implementation for wallpaper view counts like this...
(the values are there for demonstration purposes but the overall format is the same)
SADD wallpapers:100:2015-12-31 "127.0.0.1"
SADD wallpapers:100:2016-01-01 "127.0.0.1"
SADD wallpapers:100:2016-01-01 "192.168.1.1"
SADD wallpapers:100:2016-01-02 "127.0.0.1"
So that should add the IP's in the associated sets. So my question is, do they allow some sort of pattern based counts?
SCARD wallpapers:100:2016:01-01
For example the above command would return "2", as there are two IPs stored in the set, but is there a way to run something like the below command to get all counts for all the dates?
SCARD wallpapers:100:*
Actually it's easier than you've ever thought: store less specific sets to be able to get what you want.
For example, if you need wallpapers:100:* it means that you just need a set called wallpapers:100 where you store unique IP addresses there.
That is, whenever you add an IP addresses to one of specific sets (i.e. daily sets), also add it to the global set for a given wallpaper identifier.
Redis is like working with a manual index. Index your data in a way you can efficiently use it. That's all! This means that data redundancy is a good approach.
EVAL "local total = 0 for _, key in ipairs(redis.call('keys', ARGV[1])) do total = total + redis.call('scard', key) end return total" 0 wallpapers:100:*
This command returns you the total number of elements in keys wallpapers:100:*.
If you want total number of unique values from all keys combined,
EVAL "return redis.call('SUNIONSTORE', 'wallpapers:temp', unpack(redis.call('keys', ARGV[1])))" 0 wallpapers:100:*
This will return the number of unique values from all keys combined and also creates a key wallpapers:temp
You can delete this key later del wallpapers:temp
I used SUNIONSTORE for the second command.
Refer EVAL.
As per my data model, I need to store many to many relationship data items in dynamodb
Example :
I have a field called studentId and every studentId will have several subjects assigned to him.
Requirement :
So that for a given studentId, I need to store all the subjects. I would need to get all the subjects assigned to a given student.
Similary, for a given subjectId, I need to know the studentIds whom that subject has been assigned to.
am planning to store this in dynamoDb as follows :
Table1 : StudentToSubjects :
Hash Key : StudenId,
RangeKey: subjectId
so that if I query using only primaryKey, it would give me all the rows having that primary key and all the different hash keys.
Secondary Key as
secondary HashKey: subjectId
Secondary RangeKey: studentId
I wanted to know if this makes sense or the right thing to do. Or there are better ways to solve this problem.
Your Design looks OK but you need to think it through before finalizing it, let say you have implemented this design and after 10 years when you will query the table for particular subject, you will get all the students of past 10 years which you might not need (when you query using secondary table-GSI).
I would probably go with following
Student Master:
Hash Key: studentId
subjectIds (Number-set or String-set)
Subject Master:
Hash Key: subjectId
Range Key: Year
studentIds (Number-set or String-set)
Advantage of this would be you will consume less queries, for particular subject or student you will consume only 1 read (if the size is less then 4kb).
Again this is just scratching a surface think of all the queries before finalizing the Schema.
Edit: You don't need to repeat the studentId it will remain unique.
it would look something like this
studentId -> s1
subjectIds -> sub1,sub2,subN (This is set)
studentId -> s2
subjectIds -> sub3,sub4
Following is the data type link you can refer http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DataModel.html#DataModel.DataTypes
I'm attempting to create an order number for customers to use. I will have multiple machines that do not have access to the same database (so can't use primary keys and generate a unique ID).
I will have a unique string that I could use for a seed for some algorithm that will generate a unique looking alphanumeric ID # for the order number. I do not want to use this unique string as the order # because its contents would not be appropriate in appearance for a customer to use for order #.
Would it be possible to combine the use of a GUID & my unique string with some algorithm to create a unique order #?
Open to any suggestions.
If you have a relatively small number of machines and each one can have it's own configuration file or setting, you can assign a letter to each machine (A,B,C...) and then append the letter onto the order number, which could just be an auto-incrementing integer in each DB.
i.e.
Starting each database ID at 1000:
1001A // First order on database A
1001B // First order on database B
1001C // First order on database C
1002A // Second order on database A
1003A // Third order on database A
1004A // etc...
1002B
1002C
Your order table in each database would have an ID column (integer) and "machine" identifier (character A,B,C...) so in case you ever needed to combine DBs into one, each order would still be unique.
Just use a straight up guid/uuid. They take into account the mac address of the network interface to make it unique to that machine.
http://en.wikipedia.org/wiki/Uuid
You can use ids and as a primary key if you generate they id from a stored procedure (or perhaps in Oracle using a sequence).
What you have to do is make each machine generate in a different range e.g. machine a from 1 to 1million, machine B from 1000001 to 2000000 etc.
You say you have a unique string that would not be 'appropriate' to show to customers.
If it's only inappropriate and not necessary i.e. security/privacy related you could just transform it somehow. A simple example would be Rot13
But generally I too would suggest using UUID (but version 4) for random numbers. The probability for generating duplicates is extremely low and there are libraries for many programming languages available.