I recently reached this limit. The error that the cli gives is very unobvious simply stating Request contains an invalid argument and it took me quite a while to realize I had reached the maximum limit of 64KB for the firestore.rules file. Would be great if that error was a bit more obvious as it would have saved me a bunch of time.
The limits are documented here.
https://firebase.google.com/docs/firestore/security/rules-structure#security_rule_limits
After doing a bunch of searching around solutions to the 64KB limit and not finding anything I contacted support. Their guidance was somewhat helpful but in other cases a bit shocking.
I'm adding these details here for anyone else that is struggling with this issue as not much else comes up when searching on Google.
Here was their response
The given limit on the ruleset size is fixed, and it cannot be increased. However, we do offer ways to reduce the size of your ruleset, or 'lines of code' in particular.
Here are some suggestions:
1) You can define your custom functions, which can be reusable throughout the ruleset
- This will definitely save you a lot of data space, and makes it look organized, as overused conditions can be called in one place
2) If possible, reconsider the database structure by making it efficient
- This means the less the number of collections and subcollections, the less rules are written, which makes the ruleset smaller in size
- Refactor your database structure and security rules as much as possible by removing unnecessary or redundant parts
3) Minimize the use of data validation rules, and put it on application-level instead
- Not only it can reduce lines of code, but it can reduce the number of expressions evaluated to avoid reaching the given limit of 1000 expressions per request
- As much as possible, use your app logic to ensure your data is on the right character length, correct data type, meets the patter criteria, etc. You may also use your app's advanced UI elements like a password textbox, a textbox that you can limit the character length, among others
Some of these suggestions such as using functions are definitely helpful. However, i'm a bit surprised by the suggestions to restructure my database and to put validation on the application level.
In my case i've already used functions quite a lot in my database rules and have removed most of the redundancy.
Asking me to reconsider the database structure and having to overhaul my application for the sake of reducing my rules file size is a huge ask for little gain.
Minimize the use of data validation rules seems completely against the design of the Firestore database. The database is designed in such a way that you can directly connect your client to your database which removes the middle application logic. Suggesting to do this seems to go directly against the architectural nature of Firebase. I would prefer to keep building my application in this way (and want to maintain my security), so this seems like a non-option.
Does anyone else have any suggestions on how to handle this issue?
Technically, this problem does not seem too dissimilar from trying to keep javascript file size down on the web. A lot of the same approaches taken by javascript minifiers could be used to reduce the size of the firebase rules file. It would be great if Firebase provided a tool like this for us.
Out of desperation I made a simple minifier that removes whitespace and comments. In case anyone else finds themselves in a similar situation and needs a quick fix. https://github.com/brianneisler/firemin
I was wondering if it's possible to avoid triggering a child_added event when I ref.push() a value myself, while still listening for new values added by others. I couldn't find anything relevant in the docs.
I know I can detect and ignore these cases if I need to but I was actually thinking of ways to reduce bandwidth usage, given that I can already find out if/when the value was inserted correctly with push() via its onComplete callback.
A solution I would like to avoid would be reading from one place and writing to another (as in different locations inside the same database). This could work for very simple applications but would otherwise get real messy real fast.
Just FYI, I don't have any specific use-case in mind. This is more of a theoretical "what if" scenario, stuff that comes to mind as I learn about Firebase.
Thanks in advance!
I'm designing an API and I want to allow my users to combine a GET parameter with AND operators. What's the best way to do this?
Specifically I have a group_by parameter that gets passed to a Mongo backend. I want to allow users to group by multiple variables.
I can think of two ways:
?group_by=alpha&group_by=beta
or:
?group_by=alpha,beta
Is either one to be preferred? I've consulted a few API design references but no-one seems to have a view on this.
There is no strict preference. The advantage to the first approach is that many frameworks will turn group_by into an array or similar structure for you, whereas in the second approach you need to parse out the values yourself. The second approach is also less verbose, which may be relevant if your query string is particularly large.
You may also want to test with the first approach that the query strings always come into your framework in the order the client sent them. Some frameworks have a bug where that doesn't happen.
I have a system where 'dynamic logic' is implemented as Drools Rules, using Rules engine.
For each client implementation, custom pricing and tax calculation logic is implementted using the drl files for that specific implementation.
rule 'abc'
when
name = 'X'
then
price= '12'
end
And one rule's condition is dependent on what's set in the previous rules, so there is basically rules transition.
This is really painful as the drools rules are not sequential programming and is not developer friendly. There get introduced lots of bugs due to mis-interpretation of how drools evaluates.
Are there a better 'java/groovy' alternative that could easily replace it?
I think the answer is going to depend on what you ultimately want the end solution to be. If you are wanting to pull your business rules out of a rules engine and put them into java/groovy, that is very different from wanting to pull them out of one rules engine and into another.
Your questions seems to lean towards the prior, so I'll address that. Be very careful with this approach. The previous individuals who implemented this appear to have done this the proper way with respect to using the Rete algorithm, as it sounds like the firing of one rule can execute other rules, this is good business rules - they aren't sequential they are declarative. Remember that imperative software is written for engineers mostly, it doesn't map back to the real world 100% of the time :)
If you put want to move this into java/groovy you are moving into an imperative language, which could put you into an if/then/else hell. I would suggest the following:
Isolate this code away from the rest of your codebase - you will be doing a lot of maintenance on this code in the future when the business changes their rules. Good interface design and encapsulation here will payoff big time down the road.
Develop some type of DSL with your business customer so when they say something like "Credit Policy", you know exactly what they are referring to and can change the related rules appropriately.
Unit Test, Unit Test, Unit Test. This applies to your current configuration too. If you are finding bugs now, why aren't your tests? It doesn't take long to setup junit to create an object and invoke your Drools engine and test the response. If you add some loops to test ranges of variables that expect the same response you can be in the hundreds of thousands of tests in no time.
As an aside: If you don't want to go down this route, then I highly suggest getting some training on Drools so that you understand the engine and Rete if you don't already. There are some big wins you can make with your customers if you are able to quickly translate their rules into implementable software.
When I try to refactor my functions, for new needs, I stumble from time to time about the crucial question:
Shall I add another variable with a default value? Or shall I use only one array, where I´m able to add an additional variable without breaking the API?
Unless you need to support a flexible number of variables, I think it's best to explicitly identify each parameter. In most cases you can add an overloaded method that has a different signature to support the extra parameter while still supporting the original method signature. If you use an array for passing variables it just makes it too confusing for users of your API. Obviously there are some inputs that lend themselves to an array (a list of points in a polygon, a list of account IDs you wish to perform an action on, etc.) but if it's not a variable that you would reasonably expect to be an array or list, you should pass it into the method as a separate parameter.
Just like many questions in programming, the right answer is "it depends".
To take Javascript/jQuery as an example, one good rule of thumb is whether the parameter will be required each time the function is called or whether it is optional. For example, the main jQuery function itself requires an expression to determine what element(s) the operation will affect:
jQuery(expresssion)
It makes no sense to try to pass this parameter as part of an array as it will be required every time this function is called.
On the other hand, many jQuery plugins require several miscellaneous parameters that may be optional. By convention, these are passed as parameters via an 'options' array. As you said, this provides a nice interface as new parameters can be added without affecting the existing API. This makes the API clean as well since the user can ignore those options that are not applicable.
In general, when several parameters are involved, passing them as an array is a nice convention as many of them are certainly going to be optional. This would have helped clean up many WIN32 API's, although it is more difficult to deal with arrays in C/C++ than in Javascript.
It depends on the programming language used.
If you have a run-of-the-mill OO language, you should use an object that you can easily extend, if you are really concerned about API consistency.
If that doesn't matter that much, there is the option of changing the method signature and overloading the method with more / different parameters.
If your language doesn't support either and you want the API to be binary stable, use an array.
There are several considerations that must be made.
Where is the function used? - Only in code you created? One place or hundreds of places? The amount of work that will need to be done to maintain existing code is important. Remember to include the amount of time it will take to communicate to other programmers that may currently be using your function.
How critical is the new parameter? - Do you want to require it to be used? If it has a default value, will that default value break existing use of the function in any subtle ways?
Ease of comprehension - How many parameters are already passed into the function? The larger the number, the more confusing and error prone it will be. Code Complete recommends that you restrict the number of parameters to 7 or less. If you need more than that, you should try to abstract some or all of the related parameters into one object.
Other special considerations - Do you want to optimize your efforts for any special conditions such as code speed or size? Are there any special considerations that must be taken into account for your execution environment? Keep in mind your goals for the project and make sure you aren't working against them with whatever design choice you make.
In his book Code Complete, Steve McConnell decrees that a function should never have more than 7 arguments, and rarely even that many. He presents compelling arguments - that I can't cite from memory, alas.
Clean Code, more recently, advocates even fewer arguments.
So unless the number of things to pass is really small, they should be passed in an enveloping structure. If they're homogenous, an array. If not, then a reasonably lightweight object should be built for the purpose.
You should do neither. Just add the parameter and change all callers to supply the proper default value. The reason is that parameters with default values can only be at the end, and will not be able to add any more required parameters anywhere in the parameters list, without having a risk of misinterpretation.
These are the critical steps to disaster:
1. add one or two parameters with defaults
2. some callers will supply it, and some will rely on defaults.
[half a year passed]
3. add a required parameter (before them)
4. change all callers to accept the required parameter
5. get a phone call, or other event which will make you forget to change one of the instances in part#2
6. now your program compiles perfectly, but is invalid.
Unfortunately, in function call semantics we usually don't have a chance to say, by name, which value goes where.
Array is also not a proper solution. Array should be used as a connection of similar objects, upon which there's a uniform activity performed. As they say here, if it's worth refactoring, it's worth refactoring now.