My goal is to assure that all data in remote couch db will be encrypted. When I follow this example from pouch-transform docs, my data is not encrypted on remote end point after sync,
pouch.transform({
incoming: function (doc) {
encrypt(doc);
},
outgoing: function (doc) {
decrypt(doc);
}
});
when I encrypt in outgoing it does, but in this case my data encrypted locally as well. What I'm doing wrong here, isn't the point of encryption to have data encrypted in remote db? So the only way to achieve this will be to create set/get wrappers and encrypt in them? Can I detect somehow document destination in outgoing call?
isn't the point of encryption to have data encrypted in remote db?
No. As explained in the package description:
Apply a transform function to documents before and after they are stored in the database.
In other words, it only modifies the data at rest.
This plugin has exactly no effect on the data being sent to/from CouchDB--only in the way the data is stored within PouchDB itself.
If you want to encrypt documents in CouchDB, too, you need to do this at the application layer. That is, encrypt the data yourself, and store it in the document or as an attachment in encrypted form.
I've got the problem too. I do not wanna transform the data when replicating, so I've hacked in and added a replicating flag to pass through wrappered functions when replication: https://github.com/pouchdb/pouchdb/pull/7774
And modified the pouch-transform pacakge to pass the option to the outcoming callback:
{
// decrypt it here.
outgoing: async (doc, args: IPouchDBWrapperArgs, type: TransformPouchType) => {
const {options} = args;
if (!options.replicating) {
doc = await this.decryptDoc(doc, options);
}
return doc;
},
};
Related
Firebase Realtime Database Overrides my Data at a location even when I use .push() method. The little-concrete knowledge I have about writing to Firebase Realtime database is that writing to Firebase real time database can be done in few several ways. Two of the most prominent are the
set() and 2. push() method.
The long story short, push() is used to create a new key for a data to be written and it adds data to the node.
So fine, firebase has being co-operating with me in my previous projects but in this, I have no idea what is going on. I have tried different blends of push and set to achieve my goal but no progress so far.
In the code below, what I want to achieve is 2 things, write to a location chatUID, message and time only once, but write severally '-MqBBXPzUup7czdG2xCI' all under the same node "firebaseGeneratedId1" ->
A better structure is below.
Help with code. Thanks.
UPDATE
Here is my code
The writers reference
_listeningMsgRef = _msgDatabase
.reference()
.child('users')
.child(userId)
.child('chats')
.child(chatUIDConcat);
When a user hits sendMessage, here is the function called
void sendMessage() {
_messageController.clear();
var timeSent = DateTime.now().toString();
//Send
Map msgMap = {
'message': msg,
'sender': userId,
'time': timeSent,
'chatUID': chatUIDConcat
};
//String _key = _listeningMsgRef.push().key;
_listeningMsgRef.child(chatUIDConcat).set().whenComplete(() {
SnackBar snackBar = const SnackBar(content: Text('Message sent'));
ScaffoldMessenger.of(context).showSnackBar(snackBar);
DatabaseReference push = _listeningMsgRef.child(chatUIDConcat).push().set(msgMap);
});
}
The idea about the sendMessage function, is to write
chatUID:"L8pacdUOOohuTlifrNYC3JALQgh2+q5D38xPXVBTwmwb5Hq..."
message: "I'm coming"
newMessage: "true"
sender: "L8pacdUOOohuTlifrNYC3JALQgh2"
When it is complete, then push new nodes under the user nodes.
EDIT:
I later figured out the issue. I wasn't able to achieve my goal because I was a bit tensed while doing that project. The issue was I was wanted to write new data into the '-MqBBXPzUup7czdG2xCI' node without overwriting the old data in it.
The solution is straight forward. I just needed to ensure I wrote data in that node as new nodes under it. Nothing much, thanks
Frank van Puffelen for your assistance.
Paths in Firebase Realtime Database are automatically created when you write any data under then, and deleted when you remove the last data under them.
So you don't need to first create the node for the chat room. Instead, it gets auto-created when you write the first message into it with _listeningMsgRef.child(chatUIDConcat).push().set(msgMap)
Assuming that one has created a device group in Firebase Cloud Messaging, is there a way to retrieve an existing notification_key for a device group after it's been created?
Is there a way to look up the notification_key based on either notification key name or by registration id?
It seems the notification key is only returned on the create method and if the key is ever lost or errors on saving to the the database - it would be impossible to add another registration id to the the key name without a notification_key since it already exists.
You can retrieve the notification_key for a device group if you know the notification_key_name that was used when you created it. Ref: https://firebase.google.com/docs/cloud-messaging/android/device-group
Use this:
https://android.googleapis.com/gcm/notification?notification_key_name=your-key-name
Ref: https://groups.google.com/forum/#!topic/firebase-talk/ytovugx8XNs
For example:
let options = {
url: 'https://android.googleapis.com/gcm/notification?notification_key_name=the_name',
method: 'GET',
headers: {
"Content-Type": "application/json",
"Authorization": "key=" + authorizationKey,
"project_id": projectId
}
};
request(options, function (error, response, body) {
if (!error) {
res.json(body);
}
else {
res.json(error);
}
});
One thing I found when using this call was that the notification_key returned was always different, but I was able to use it successfully to add or remove registration_ids.
I'll add some additional information from comments I made earlier:
I've had quite a bit of trouble with device groups. The only way to delete a device group is to remove all the notifications keys it contains. If you lose track of a notification key that you have added to the device group then deleting that device group is impossible, as there is no way currently to get a list of the notification keys in a device group.
The device group is a very good mechanism for sending messages to multiple devices (see #AL.'s comment for another method). I also store in the database the registration_ids for each device group. The reason for that is if the user deletes their account for my app I also delete their device group so that the device group name can be reused.
There is currently no API to retrieve the Device Groups (notification_keys) associated with a given Registration Token. AFAIK, managing/mapping relationships of Device Groups and its associated registration tokens are the developers responsibility.
For your scenario, I would suggest to temporarily store the notification_key until it is successfully stored in your App Server.
Some possibly helpful posts:
Firebase Cloud Messaging - Managing Registration Tokens
Managing FCM device groups
I've read the Firebase docs on Stucturing Data. Data storage is cheap, but the user's time is not. We should optimize for get operations, and write in multiple places.
So then I might store a list node and a list-index node, with some duplicated data between the two, at very least the list name.
I'm using ES6 and promises in my javascript app to handle the async flow, mainly of fetching a ref key from firebase after the first data push.
let addIndexPromise = new Promise( (resolve, reject) => {
let newRef = ref.child('list-index').push(newItem);
resolve( newRef.key()); // ignore reject() for brevity
});
addIndexPromise.then( key => {
ref.child('list').child(key).set(newItem);
});
How do I make sure the data stays in sync in all places, knowing my app runs only on the client?
For sanity check, I set a setTimeout in my promise and shut my browser before it resolved, and indeed my database was no longer consistent, with an extra index saved without a corresponding list.
Any advice?
Great question. I know of three approaches to this, which I'll list below.
I'll take a slightly different example for this, mostly because it allows me to use more concrete terms in the explanation.
Say we have a chat application, where we store two entities: messages and users. In the screen where we show the messages, we also show the name of the user. So to minimize the number of reads, we store the name of the user with each chat message too.
users
so:209103
name: "Frank van Puffelen"
location: "San Francisco, CA"
questionCount: 12
so:3648524
name: "legolandbridge"
location: "London, Prague, Barcelona"
questionCount: 4
messages
-Jabhsay3487
message: "How to write denormalized data in Firebase"
user: so:3648524
username: "legolandbridge"
-Jabhsay3591
message: "Great question."
user: so:209103
username: "Frank van Puffelen"
-Jabhsay3595
message: "I know of three approaches, which I'll list below."
user: so:209103
username: "Frank van Puffelen"
So we store the primary copy of the user's profile in the users node. In the message we store the uid (so:209103 and so:3648524) so that we can look up the user. But we also store the user's name in the messages, so that we don't have to look this up for each user when we want to display a list of messages.
So now what happens when I go to the Profile page on the chat service and change my name from "Frank van Puffelen" to just "puf".
Transactional update
Performing a transactional update is the one that probably pops to mind of most developers initially. We always want the username in messages to match the name in the corresponding profile.
Using multipath writes (added on 20150925)
Since Firebase 2.3 (for JavaScript) and 2.4 (for Android and iOS), you can achieve atomic updates quite easily by using a single multi-path update:
function renameUser(ref, uid, name) {
var updates = {}; // all paths to be updated and their new values
updates['users/'+uid+'/name'] = name;
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.once('value', function(snapshot) {
snapshot.forEach(function(messageSnapshot) {
updates['messages/'+messageSnapshot.key()+'/username'] = name;
})
ref.update(updates);
});
}
This will send a single update command to Firebase that updates the user's name in their profile and in each message.
Previous atomic approach
So when the user change's the name in their profile:
var ref = new Firebase('https://mychat.firebaseio.com/');
var uid = "so:209103";
var nameInProfileRef = ref.child('users').child(uid).child('name');
nameInProfileRef.transaction(function(currentName) {
return "puf";
}, function(error, committed, snapshot) {
if (error) {
console.log('Transaction failed abnormally!', error);
} else if (!committed) {
console.log('Transaction aborted by our code.');
} else {
console.log('Name updated in profile, now update it in the messages');
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.on('child_added', function(messageSnapshot) {
messageSnapshot.ref().update({ username: "puf" });
});
}
console.log("Wilma's data: ", snapshot.val());
}, false /* don't apply the change locally */);
Pretty involved and the astute reader will notice that I cheat in the handling of the messages. First cheat is that I never call off for the listener, but I also don't use a transaction.
If we want to securely do this type of operation from the client, we'd need:
security rules that ensure the names in both places match. But the rules need to allow enough flexibility for them to temporarily be different while we're changing the name. So this turns into a pretty painful two-phase commit scheme.
change all username fields for messages by so:209103 to null (some magic value)
change the name of user so:209103 to 'puf'
change the username in every message by so:209103 that is null to puf.
that query requires an and of two conditions, which Firebase queries don't support. So we'll end up with an extra property uid_plus_name (with value so:209103_puf) that we can query on.
client-side code that handles all these transitions transactionally.
This type of approach makes my head hurt. And usually that means that I'm doing something wrong. But even if it's the right approach, with a head that hurts I'm way more likely to make coding mistakes. So I prefer to look for a simpler solution.
Eventual consistency
Update (20150925): Firebase released a feature to allow atomic writes to multiple paths. This works similar to approach below, but with a single command. See the updated section above to read how this works.
The second approach depends on splitting the user action ("I want to change my name to 'puf'") from the implications of that action ("We need to update the name in profile so:209103 and in every message that has user = so:209103).
I'd handle the rename in a script that we run on a server. The main method would be something like this:
function renameUser(ref, uid, name) {
ref.child('users').child(uid).update({ name: name });
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.once('value', function(snapshot) {
snapshot.forEach(function(messageSnapshot) {
messageSnapshot.update({ username: name });
})
});
}
Once again I take a few shortcuts here, such as using once('value' (which is in general a bad idea for optimal performance with Firebase). But overall the approach is simpler, at the cost of not having all data completely updated at the same time. But eventually the messages will all be updated to match the new value.
Not caring
The third approach is the simplest of all: in many cases you don't really have to update the duplicated data at all. In the example we've used here, you could say that each message recorded the name as I used it at that time. I didn't change my name until just now, so it makes sense that older messages show the name I used at that time. This applies in many cases where the secondary data is transactional in nature. It doesn't apply everywhere of course, but where it applies "not caring" is the simplest approach of all.
Summary
While the above are just broad descriptions of how you could solve this problem and they are definitely not complete, I find that each time I need to fan out duplicate data it comes back to one of these basic approaches.
To add to Franks great reply, I implemented the eventual consistency approach with a set of Firebase Cloud Functions. The functions get triggered whenever a primary value (eg. users name) gets changed, and then propagate the changes to the denormalized fields.
It is not as fast as a transaction, but for many cases it does not need to be.
I need to create the notifications for future events. for example similar to reminders. I was thinking that I can use the the server timestamp + future time as the key and than pop only the notifications with the timestamp older than current server time. However I need the server time.
I can't use client side time stamp as it might be out of sync.
Is there a way to get the current time stamp from firebase?
I know that there is a placeholder for timestamp which is replaced on the server side. Can I use this as a key?
I can imagine that probably not so is there a way to listen to this event when the placeholder is replaced with real timestamp?
or generally is there any other sensible method around this problem?
You're probably solving the X/Y problem here, and you might be better off explaining your use case and getting a better overall solution; fetching a server timestamp to use on the client seems extremely likely to be a conceptual problem.
For example, if my goal is just to display the time in messages, I can simple call set, monitor the path, and display what it returns:
var ref = new Firebase(URL);
ref.on('child_added', function(snap) {
// client just listens on path, records have serve timestamp when they arrive
console.log('the last event was added at', snap.val().time);
});
ref.push({ name: 'Kato', time: Firebase.ServerValue.TIMESTAMP });
If you are queuing future events, just store them in a different path and move them to the "present" path when ready. Rather than trying to fetch "now" and insert it later.
If you are set on fetching the timestamp, the simplest way would be to set up a dummy path and set the value against it:
new Firebase(URL).transaction(function(currValue) {
return Firebase.ServerValue.TIMESTAMP;
}, function(err, success, snap) {
console.log('the current server timestamp', snap.val());
});
first excuse my writing, I'm using google translator.
What I want is to create users meteor, but only from the server and not allow the client to create users.
try putting
Accounts.validateNewUser (function () {
return false;
});
but it denies me create users to the server (which I do not want)
thank you very much in advance.
You can use Accounts.config for that purpose. Just anywhere in your server code do
Accounts.config({
forbidClientAccountCreation: true,
});
One way to do this would be to add some kind of un-guessable string to the profile key when you create a user on the server, check for that key in the validateNewUser function and then remove it immediately using observe on an appropriate cursor.
On the server you would have:
Accounts.validateNewUser(function(user) {
return (user.profile && user.profile.createUserKey === [KEY]);
});
Meteor.users.find().observe({
added: function(user) {
Meteor.users.update(user, {$unset: {'profile.createUserKey': true}});
}
});
And then your server-side createUser line looks like this (along with any other data you want to pass in the profile):
Accounts.createUser({username: [USERNAME], password: [PASSWORD], profile: {createUserKey: [KEY]}});
Assuming you don't publish the validation function anywhere (like Github), a client will have no way of knowing what the key is and would thus be unable to create a new user. If you don't trust yourself not to push it to Github or similar, store the key in an unpublished collection in your MongoDB and then pull it out on server start-up - that's what I do.
NOTE I think it's slightly unsatisfactory that I'm using a cursor here as it would be far better to remove the key in an onCreateUser callback, but this seems to be called before validateNewUser, so it's no use here. The alternatives are using your own modified version of the createUser function in the Accounts package, which probably isn't too tough but is a little involved to go into here, or just leaving the key in the user document and making sure you don't publish it.