I have a Bokeh script which calls the data using a function wrapped with joblib's #memory.cache decorator. When I run the script as a python script the get_data function is fast (cached). When I call it using bokeh server --show code.py it seems like cache is lost and the function is re-evaluated, making data retrieval slow. How can I make Bokeh work nicely with Joblib?
It's hard to say for certain without being able to run an example that reproduces what you are seeing. But my guess is that it has something to do with the way the Bokeh server code runner executes the app script, on every session.
So, I can think of a few possible things to try.
First, as of 0.12.4 there's examples and guidance for embedding a Bokeh server as a library e.g. in a standalone python script, or in a Flask or Tornado app. The examples there all also use FunctionHandler which does not exec. My hunch is that this is more like the standard single process/single namespace python execution model, and will play better with your joblib decorator.
(If you try this route, and it works, please let use know somehow, it's probably worth documenting better.)
Otherwise, another option that might work better is to use lifecycle hooks to provide your wrapped function in a way that is sure to be shared across sessions. You can see this technique in the spectrogram example (c.f. the audio.py)
Finally, just some gentle advice for SO. If you can include a minimal example code, that greatly increases the odds of being able to get code back in an answer. E.g., if there was example code here that I could try to get working, then I'd be able to post a complete working code in the answer.
Related
I'm currenty cleaning up my first big R project and at a point, where I have a lot of functions implemented but I am not sure, which function got called and used by me in an other script and which function got never used. So now I want to get all calls of this function in my project. Is this possible?
I'm using RStudio and a lot of other IDEs I've used got a feature like this, so I was wondering if this is also implemented in RStudio.
I searched the web and stack overflow, but got no answer, so I assume that this is not possible but I wanted to ask, just in case it IS possible but I didn't found the right answer.
Thank you!
You can use multithreading in Python and send lots of http requests, like in this SO question. My question is, is there any easy way to do this in R? I've seen a guide for RCurl here, but I'd prefer a simpler solution if possible. Currently I'm looping through a series of ids, it's be great to send all (or more) of them at once.
That guide to multiple requests in Rcurl looks pretty simple, in fact I'd say it looks simpler to me than the solution to the Python question you've linked. Better yet, the work is already done for you. Most of that guide is going into detail about the advantages of concurrent requests; the method itself is deceptively simple, and is provided for you pre-cooked right at the top of the page.
You can literally cut and paste the code shown at the top of the post into an R script (include library(RCurl) above it), run that code to source the function, then call the function with a single line.
I won't paste the function code here, since you should get that from its author, but once you've sourced that function, their example usage is:
uris = c("http://www.omegahat.org/index.html", "http://www.omegahat.org/RecentActivities.html")
z <- getURIs(uris)
I just did the above on my own computer, and it works perfectly. I'd be surprised if you can find a simpler solution than that.
I was wondering if it is possible to use Julia to perform computations on a webpage in an automated way.
For example suppose we have a 3x3 html form in which we input some numbers. These form a square matrix A, and we can find its eigenvalues in Julia pretty straightforward. I would like to use Julia to make the computation and then return the results.
In my understanding (which is limited in this direction) I guess the process should be something like:
collect the data entered in the form
send the data to a machine which has Julia installed
run the Julia code with the given data and store the result
send the result back to the webpage and show it.
Do you think something like this is possible? (I've seen some stuff using HttpServer which allows computation with the browser, but I'm not sure this is the right thing to use) If yes, which are the things which I need to look into? Do you have any examples of such implementations of web calculations?
If you are using or can use Node.js, you can use node-julia. It has some limitations, but should work fine for this.
Coincidentally, I was already mostly done with putting together an example that does this. A rough mockup is available here, which uses express to serve the pages and plotly to display results (among other node modules).
Another option would be to write the server itself in Julia using Mux.jl and skip server-side javascript entirely.
Yes, it can be done with HttpServer.jl
It's pretty simple - you make a small script that starts your HttpServer, which now listens to the designated port. Part of configuring the web server is that you define some handlers (functions) that are invoked when certain events take place in your app's life cycle (new request, error, etc).
Here's a very simple official example:
https://github.com/JuliaWeb/HttpServer.jl/blob/master/examples/fibonacci.jl
However, things can get complex fast:
you already need to perform 2 actions:
a. render your HTML page where you take the user input (by default)
b. render the response page as a consequence of receiving a POST request
you'll need to extract the data payload coming through the form. Data sent via GET is easy to reach, data sent via POST not so much.
if you expose this to users you need to setup some failsafe measures to respawn your server script - otherwise it might just crash and exit.
if you open your script to the world you must make sure that it's not vulnerable to attacks - you don't want to empower a hacker to execute random Julia code on your server or access your DB.
So for basic usage on a small case, yes, HttpServer.jl should be enough.
If however you expect a bigger project, you can give Genie a try (https://github.com/essenciary/Genie.jl). It's still work in progress but it handles most of the low level work allowing developers to focus on the specific app logic, rather than on the transport layer (Genie's author here, btw).
If you get stuck there's GitHub issues and a Gitter channel.
Try Escher.jl.
This enables you to build up the web page in Julia.
I'm stuck using the 4.0 version of lua which does not seem to support the os library. Is there a way to include this library into my project?
Or get another way to use the functionality contained within pertaining to date time calculations?
Preferably by using a *.lua file and not a *.c file since I don't have complete access to the code.
When I run the following line,
print(os.time{year=1970, month=1, day=1, hour=0})
I get an error stating:
attempt to index global 'os'(a nil value)
As #Colonel Thirty Two said it's not possible to use the os library. So the time() funciton is not available for me.
Adding to the (totally correct) currently accepted answer (that if "os" access was not allowed to you, you're generally done), there's some very slight chance the Original Programmer may have provided you with some alternative facilities to do your thing (fingers crossed). In a perfect world, those would be described in some kind of a User's Manual for your scripting environment. But if the manual was lost to time (or never existed in the first place), you might possibly try your luck at exploring any preloaded libraries by digging through the result of the globals() Basic Function. (At least I hope that's how it was done in 4.0 too.) That is, if the Original Programmer didn't block globals() for you too...
I'm building a custom SSIS data flow component that, as part of its process, takes multiple inputs and rolls them into a single output (along the lines of what the UNION ALL component does). I'm having a little bit of trouble implementing the runtime methods and having been looking for source code for asynchronous transformations that use multiple inputs that I can use as a model for my own code.
Does anyone know of available source code or have their own code they'd be willing to share with me to help get me on the right track? Much appreciated!
Off the top of my head, I'd look at the Programming Samples from MS's IS team as well as the Community Transform. Some of those are open source so if you can find one that's async, you should be able to see what you're missing.
My half-remembered logic is that you'll need to look at all the InputCollections for a given IDTSComponentMetaData100 and then work with that data.