I have a MongoDB collection Foods in my Meteor app with about 8000 entries with almost 1000 fields each.
For some reason, since I included it, the response time when I call a server method from the client is very slow (seconds). For debugging, I've been removing things one by one. Now, I don't use the collection in any of the functions involved (I've even replaced the server method with just a console.log), and yet if I add the line import { Foods } from '../imports/collections.js'; to the server, the response is slow, and if I don't it is fast.
Does anyone have any idea why this could be?
Note: OP has already answered his question satisfactory. However, the following information should be pointed out, especially for those who face similar issues in context of controlling Meteor's autopublish behavior.
As long as autopublish exists in your packages list, each collection publishes to the client at the very moment, your Mongo.Collection is created.
See this code of the Mongo.Collection constructor.
Autopublish is then triggered, even just by importing your file that contains a Mongo.Collection constructor to be called (which is very likely your case).
However, you dont need to remove the autopublish package if you only want this one collection to prevent auto publishing.
Your Mongo.Collection constructor accepts as second parameter an object with options. To prevent auto publish only for this collection (while having autopublish active) you can add the following option:
{
_preventAutopublish: true,
// ... other options if desired
}
An example for your given Foods collection could be
export const Foods = new Mongo.Collection('foods', {_preventAutopublish: true});
Although the option is not documented, it works and comes in very handy when prototyping.
However, keep in mind that autopublish is not secure and should never be present in a release that is expected to be deployed on a server.
So the answer was easy.
As said in the Meteor docs,
By default, Meteor automatically publishes every document in your collection to each connected client. To turn this behavior off, remove the autopublish package
So, simply by importing the collection, Meteor interprets that you want to publish it all, and that is what was making it slow, even if I wasn't explicitly using it.
Related
I have a collection that needs to be updated. There's a need to add new field and fill it out based on the existing field.
Let's say I have a collection called documents:
documents/{documentId}: {
existingField: ['foo', 'bar'],
myNewField ['foo', 'bar']
}
documents/{anotherDocumentId}: {
existingField: ['baz'],
myNewField ['baz']
}
// ... and so on
I already tried to fire up local cloud function from emulator that loops for each document and writes to production data based on the logic I need. The problem is that function can only live up to max of 30 seconds. What I need would be some kind of console tool that I can run as admin (using service-account) to quickly manage my needs.
How do you handle such cases?
Firebase does not provide a console or tool to do migrations.
You can write a program to run on your development machine that uses the one of the backend SDKs (like the Firebase Admin SDK) to query, iterate, and update the documents and let it run as long as you want.
There is nothing specific built into the API for this type of data migration. You'll have to update each document in turn, which typically involves also reading all documents (or at least their IDs).
While it is possible to do this on Cloud Functions, I find it easier to do it with a local Node.js script, as that doesn't have the runtime limits Cloud Functions imposes.
Hello I am trying to read a module with this code:
(: Entry point - must be a read-only query. :)
xdmp:invoke(
'/path/mydocument.xqy',
(xs:QName('var1'), 'test',
xs:QName('var2'), "response"))
I am new in MarkLogic, I am using groovy and the api to connect to it, but also I saw I can invoke the module with this and indeed I did but it returns me
your query returned an empty sequence
I want to know if I can query xs:QName('var1'), 'test', changing test with a wildcard or how can I get all the information from the file called /path/mydocument.xqy?
I tried to use this:
xdmp:document-get("/path/mydocument.xqy)
but it says the file is not found. Although, if I use invoke I can query it, but I don't know what are the values I have to pass. I was wondering if there is something like sql using %% or something to give me all the data.
To answer the first question: "I am trying to read a module "
IF the module is in the database, then you must query the Modules database in which the module resides.
If the module is in the filesystem then you cannot directly access its source as a document but you can by executing xdmp:filesystem-file()
Simplification:
With the Default configuration of the server and REST client, user placed modules are in the "Modules" database and user placed documents are in the "Documents" database. This means, if you do a GET (read a "Document") with no additional parameters, it will return documents from the "Documents" database. Assuming you are using the default configuration for client and server, this would result in the behavior you are seeing. E.g. your Module code is in the Modules database, doing a GET for it by name will search the Documents database and correctly not find it.
You don't mention, and I don't know, the groovy library being used, but the REST API itself and all implementations of general purpose ML REST client libraries I am familiar with have options for overriding the default database with another. If the groovy library supports that, then specify the "Modules" database for your query and it should return the module document. Note: content-type will be application/text not text/xml.
You can simplify things for testing by bypassing the libraries and simply use a browser and try a URL like this http://yourserver.com:8000/v1/documents?uri=/your/module.xqy&database=Modules
Ref: https://docs.marklogic.com/REST/GET/v1/documents
Making the appropriate changes to the path and server for your use.
If you are still confused, then you should start with the basic MarkLogic tutorials and work through them one by one. You will most likely succeed faster by doing this then jumping straight into coding you don't understand yet.
DETAIL:
Note: The default behaviour is to EXECUTE documents when doing a GET call, using the Modules database. Thus doing a GET of http://yourserver:8000/your/module.xqy will EXECUTE it not return its source.
You will notice the REST API has a uri query parameter. This is EXECUTING the REST API code on /v1/documents which in turn will read the document specified by the uri and database parameters and return it.
I guess I can use:
xdmp:invoke(/pview/get-pview-browse-profiles.xqy,
cts:and-query((
cts:element-value-query(
xs:QName("letter"),"*", "wildcarded"),
cts:element-value-query(
xs:QName("collection"),"*", "wildcarded"))))
although it doesn't return anything
It seems that most of the reactivity is on the client side.
I've got a server function that I need to run based on the number of items in a collection. The function creates a schema for an OrderSubmissions collection based on the number of documents in Services, which changes.
On the client this is easy - I simply do
/lib/schemas
if(Meteor.isClient){
Tracker.autorun(function(){
Meteor.subscribe('services', function(){
// re-creates OrderSubmissions schema any time the Services subscription returns different data
});
});
};
I could also rig up something with Session or ReactiveVar. Unfortunately, all of these techniques are only available to the client. I need to do:
/lib/schemas
if(Meteor.isServer){
// re-creates OrderSubmissions schema any time Services collection changes
};
Is the only way to do this to use .observe? It seems like kind of an expensive thing to do. My Services collection will change very very infrequently (in fact, at this point all I want to do is create the OrderSubmissions schema in /lib/schemas when my /server/fixtures.js is done loading stuff into the Services collection.)
Unfortunately for my case, Meteor loads stuff in /lib before /server, so my schema in /lib is being created erroneously:
/lib/schemas.js runs and OrderSubmissions schema, which depends on Services to be populated, gets created erroneously on both the server and client because there is nothing in the Services collection.
/server/fixtures.js runs and populates the Services collection.
I need #2 to happen before #1, but #2 needs to stay in server code. I don't want to wrap it in if(Meteor.isServer) in /lib since it's not secure.
If you add peerlibrary:server-autorun
You could do the following on the server
Tracker.autorun(function(){
// this code now reruns on the server if there is a reactive dependency enclosed here
})
i'm using accounts-vk for providing login through vkontakte.ru
All works perfectly before i'm deleted meteor-autopublish package.
My connectSubmit.js:
Template.connectSubmit.events({
'submit form': function(e) {
e.preventDefault();
var query = {
vk_id: Meteor.user().services.vk.id,
user_id: Meteor.userId(),
photo: Meteor.user().services.vk.photo
};
query._id = Connects.insert(query);
Router.go('index');
}
});
Error is:
Uncaught TypeError: Cannot read property 'vk' of undefined
And second problem with my template:
This worked good before i'm delete autopublish:
{{currentUser.services.vk.first_name}}
But now not working.
I'm think problem with Meteor.publish function, but i'm have no idea how to resolve it.
When autopublish is on, it will publish all of your user's fields. When you remove autopublish only a few fields are published (username, _id, emails, profile).
If you publish the services field, you are exposing things like login tokens and hashed passwords to the client. Obviously, you never want to do that in production (which is why autopublish should always be removed). For more on this, see the "published secrets" section of my common mistakes article.
So the short answer to your question is that the client shouldn't be doing any of these things in the first place.
The long answer is that if you have logic which requires these data, you'll need to do one of two things (which may warrant separate questions):
if the services info is only needed for data mutation
Use a server-side method in this case. In your submit event you would call a method which does a findOne on the current user and updates the Connects collection. Here it's safe to read the services data because the code is running on the server.
if the services info is needed on the client (e.g. a photo)
You'll need to copy the data from services into a safe field like profile. You could do this either when the user creates her account or whenever she logs in.
Does anyone know how to drop a collection safely in Meteor? There doesn't seem to be an API method for this. The collections are published, so I want to cleanup the associated data and functions (besides just removing the MongoDB collection).
Someone asked a similar question but no one actually answered it (just other concerns apart from dropping the collection).
Clarification: I need to do this from the meteor program at runtime.
You can drop a collection with the basic mongo shell command : db.collection.drop()
To do that, you have to connect to your mongo shell in your command line.
There is another easier way : Robomongo ( http://robomongo.org/ ) which is a really nice GUI to manage mongo db. Once logged on mongo, just right click on your collection and then > drop (and confirm of course).
For example, to log in on your local environment, give any name and skip login. The address should be localhost : 3001.