Let's say I have the current Firestore data:
COLLECTION_1
DOC_A
DOC_B
DOC_C
COLLECTION_2
DOC_X
And I have some exported data, which is:
COLLECTION_1
DOC_B // DOC_A IS NOT HERE
DOC_C
DOC_D // DOC_D IS AN "EXTRA"
// COLLECTION_2 IS MISSING
COLLECTION_3 // THERE IS AN "EXTRA" COLLECTION
DOC_3
What happens when I import data data? Will some kind of "merge" occur, or my entire Firestore will be replaced with the imported data as is? I'm gessing DOC_D and COLLECTIO_3 will be added for sure. But will DOC_A and COLLECTION_2 (the ones missing on the exported data) still be there after the import?
PS: I'm asking this on the context of a full import (i.e: all collections).
To import specific collections, it seems that you need to use an export that exported specific collections, and you need to import it back using gcloud. The web console only allows for full imports. Anyway, I think the global "merge or override" behavior on a full import is probably similar to the same behavior over a specific collection.
Docs on import/export
This was super quick to test it. The result is
COLLECTION_1
DOC_A
DOC_B
DOC_C
DOC_D
COLLECTION_2
DOC_X
COLLECTION_3
DOC_3
Everything is happening by document. Mentioned documentation says:
If a document with the same ID already exists, the import overwrites
the existing document.
If a document in your database is not affected by an import, it will
remain in your database after the import.
And this exactly happening. Documents that are exist in the import are overridden, and not existing are added. If document is in Firestore but not in import it remains unchanged.
BTW if you test it in GUI, you have to refresh the page after import. Otherwise changes are not visible (at least it happened to me).
Related
I read that Openrefine Wikidata import can only create statements with rank = Normal.
I am in a situation where I would like to import deprecated identifiers, namely old ISSNs (identifiers for serial publications) that used to be used to refer to a publication, but that are no longer in use. They have value because other bibliographic databases might still use them to refer to a publication.
So I'd like to be able to import these values, and mark them as "deprecated".
OpenRefine Wikidata import really really can't do that ? any workaround I could use ?
If I can't do that by the time I import the data from OpenRefine, I can flag these statements with "reason for deprecation = withdrawn identifier value". Can I run another automated process after import that would automatically set a deprecated rank on these statements ?
If I can't set them automatically to deprecated neither at import time nor after, which other qualifier can I use to indicated these identifiers are no longer in use ? (the official term being "Cancelled ISSN")
Thanks
A possibility I found would be to use the Wikidata-Toolkit library, in which StatementBuilder can create Statements with a specified rank. I would still to know/fetch the list of statements to modify...
I am still open to suggestions on the possibility to specify ranks at import time.
i am trying to update my firebase but is not appear what i updated
this is my firebase data :
and then i exported my data :
then i updated and added another class :
and return a message is all updated :
but the problem is my data don't updated and stay in same data :
and this is my rules :
please help me to know what is and why i cant success to update that
thanks alot
Firebase automatically creates keys for any values you write, and it automatically deletes keys that no longer have any values under them.
Since you specified no value for the game key, it is immediately deleted after you add it.
If you want to make sure the key gets written when you import the JSON, be sure to give it a value. E.g.
"game": true
But note that you don't need to pre-create keys like this, as Firebase will automatically create the game key when your app writes a value somewhere under it.
In Meteor 1.4, what's the difference between these two import statements in the client code and in the server code? Why?
Client code:
import { Items } from '../imports/api/items.js';
Server code:
import '../imports/api/items.js';
Both statements will execute the file given, however the first one will add Item to the scope. This can be useful if the file has many exports and you are only concerned with that single class/function/object/etc.
The second is used for side effects provided from importing it (like initializing a store or something like that; I confess I don't know much about meteor).
Mozilla has a great resource on the import statement https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import
I'm just getting started with Flow, trying to introduce it into an existing Node codebase.
Here are two lines Flow complains about:
import Module from 'module';
const nodeVersion = Number(process.versions.node.split('.')[0]);
The warnings about these lines are, respectively:
module. Required module not found
call of method `split`. Method cannot be called on possibly null value
So it seems like Flow isn't aware of some things that are standard in a Node environment (e.g. process.versions.node is guaranteed to be a string, and there is definitely a Node builtin called module).
But then again, Flow's configuration docs suggest it's Node-aware by default. And I have plenty of other stuff like import fs from 'fs'; which does not cause any warning. So what am I doing wrong?
Module fs works as expected because Flow comes with built-in definitions for it, see declare module "fs" here: https://github.com/facebook/flow/blob/master/lib/node.js#L624
Regarding process.versions.node, you can see in the same file that the versions key is typed as a map of nullable strings, with no mention of the specific node property: versions : { [key: string] : ?string };. So you'll need to either make a PR to improve this definition, or adjust your code for the possibility of that value being null.
I guess the answer about module "module" is obvious now – there are no built-in definitions for that module in Flow in lib/node.js. You could write your own definitions, and optionally send a PR with them to the Flow team. You can also try searching github for these, someone might have done the work already.
That lib directory is very useful by the way, it has Flow definitions for DOM and other stuff as well.
I'm trying to find an example on how to use the discard-document function of Saxon. I have about 50 files 40mb each, so they are using about 4,5GB of memory in my xquery script.
I've tried to use saxon:discard-document(doc("filename.xml")) after every call to the XML file, but maybe this is not the correct way to do it? There is no difference in memory usage after using that.
I also found some questions about its usage (7 years ago), and they were suggesting running the xpath using discard-document. But I have many calls to that document, so I would have to replace all declarations with saxon:discard-document(doc("filename.xml"))/xpath/etc/etc/etc
Thanks
I think this is a good question and there is not much information available so I will try to answer it myself.
Here is an example on how to use saxon:discard-document:
declare function local:doStuffInDocument($doc as document-node()) {
$doc//testPath
};
let $urls := ("http://url1", "http://url2")
let $results :=
for $url in $urls
let $doc := saxon:discard-document(doc($url))
return local:doStuffInDocument($doc)
return $results
By using a similar code I managed to reduce the memory consumption from 4+GB to only 300MB.
To understand what discard-document does, here is a great comment from Michael Kay found at the SF maillist:
Just to explain what discard-document() does:
Saxon maintains (owned by the Transformer/Controller) a table that
maps document URIs to document nodes. When you call the document()
function, Saxon looks to see if the URI is in this table, and if it
is, it returns the corresponding document node. If it isn't, it reads
and parses the resource found at that URI. The effect of
saxon:discard-document() is to remove the entry for a document from
this mapping table. (Of course, if a document is referenced from this
table then the garbage collector will hold the document in memory; if
it is not referenced from the table then it becomes eligible for
garbage collection. It won't be garbage collected if it's referenced
from a global variable; but it will still be absent from the table in
the event that another call on document() uses the same URI again.)
And another one from Michael Kay found at the Altova maillist:
In Saxon, if you use the doc() or document() function, then the file
will be loaded into memory, and will stay in memory until the end of
the run, just in case it's referenced again. So you will hit the same
memory problem with lots of small files as with one large file -
worse, in fact, since there is a significant per-document overhead.
However, there's a workaround: an extension function
saxon:discard-document() that causes a document to be discarded from
memory by the garbage collector as soon as there are no more
references to it.
It's probably useful to understand what actually happens below the covers. The doc() function looks in a cache to see if the document is already there; if not, it reads the document, adds it to the cache, and then returns it. The discard-document() function looks to see if the document is in the cache, removes it if it is, and then returns it. By removing the document from the cache, it makes it eligible for garbage collection when the document is no longer referenced. If using discard-document has no effect on memory consumption, that's probably because there is something else still referencing the document - for example, a global variable.