I heard about celery and I really like it. But now I'm writing an application with node.js and I have to manage (asynchronous) tasks and I want to use celery to this. I've installed it in my development environment and played around with some python scripts. It all works well but I want to "call" the tasks with node.js. Has anyone tried to do something like this (with any programming language)?
I saw this example, but the base of this HTTP Gateway idea is a django application and I don't want to create a django app to only handle these calls.
I thought about creating a SimpleXMLRPCServer and use the node-xmlrpc module to connect with that. What do you think? There is a better way to do this? Is there another app or service that works natively with node.js?
Thanks in advance.
Celery will force you to inherit a whole Python stack for a simple message queue - seems like a messy pain to me. Check out coffee-resque for a simple and native solution.
Related
I'm thinking about creating an API with Rscript doing all my stuff.
But, we know that R is single-thread, and plumber also is.
Someone know a way to create an API multi thread in R? I dont think that an single-thread can help me in my case, I may have many users using my model in prod, then im afraid to use plumber.
I already see people saying about using RServer and Java to create an API multi thread. (but i dont know how do this).
Any suggestion, or links about this discussion is welcome.
Thank you all!
Just to document things that are possible to other people that may have the same question, and dont know how do this, I will put here some links that i found that can be helpful.
I dont test any idea yet to say which one is faster, or cheaper, or optimized...
But, what i found is:
You can use https://restrserve.org/ that is an alternative to plumber, even when plumber was just single-thread, restrserve already was helpful to create APIs multi-thead.
8 days ago, plumber release the version 1.0.0, that can support APIs multi-thread. Link to release: https://www.rplumber.io/news/index.html#plumber-router
Even in the past, without plumber v1.0.0, or if you dont want use the RestRserve, you could create an API single-thread with the plumber (in the past), use the Docker to Build your API and then use Kubernetes to manage the requests, creating "copys" of your API, and then the Kubernetes manage the requests, choosing the copy_API that will be used.
An talk at RSTUDIO::CONF2020 about an model that has 1.000.000 acces per day, with an R API: https://rstudio.com/resources/rstudioconf-2020/we-re-hitting-r-a-million-times-a-day-so-we-made-a-talk-about-it/
There is a piece of code, a webserver backend, thats written as a whole bunch of azure functions - and predictably its terribly slow - kind of the mistake that someone has pointed out in this blog - https://blog.maartenballiauw.be/post/2019/10/02/dont-use-azure-functions-as-a-web-application.html
Is there an easy way to convert an Azure functions based web server to a traditional simple .NET server ? I mean "magically" without rearchitecting the whole thing. Sorry I am a newbie to this whole web/azure thing and trying to find my way around an existing code base
As mentioned in comments, there isn't a easy way to convert the function to a web app directly. But I think it is not different for us to implement it. You just need to copy the copy the body of your function code and create some function or apis in your web app with the code which you copied from function. You may need to adjust some parameters in your apis in web app.
Apart from this, you can also call these functions in your new app code(without rewrite the function code in your new app). I think the whole process will not be difficult.
I am wondering if it is feasible to deploy wordpress as a series of lambda functions on AWS API gateway. Any pointers on the feasibility/gotchas would be greatly appreciated!
Thanks in advance,
PKK
You'll have a lot of things to consider with persistence and even before that, Lambda doesn't support PHP. I'd probably look at Microsoft Azure Functions instead that do support PHP and do have persistent storage.
While other languages (such as Go, Rust, Swift etc.) can be "wrapped" to run in AWS Lambda with relative ease, compiling PHP targeting the same platform and running it is a bit different (and certainly more painstaking). Think about all the various PHP modules you'd need for starters. Moreover, I can't imagine performance will be as good as something like a Go binary.
If you can do something clever with the Phalcon framework and come up with an easy build and deploy process, then maayyyybee.
Though, you'd probably need to really overhaul something like WordPress which was not designed for this at all. It still uses some pretty old conventions due to the age of the project and while that is all well and good for your typical PHP server, it's a different ball game in the sense of this "portable" PHP installation.
Keep in mind that PHP sessions are relied upon as well and so you're going to need to move those elsewhere due to the lack of persistence with AWS Lambda. You can probably find some sort of plugin for WordPress that works with Redis?? I have to imagine something like that has been built by now... But there will be many complications.
I would seriously consider using Azure Functions to begin with OR using Docker and forgoing the pricing model that cloud functions offers. You can still find some pretty cheap and scalable hosting out there.
What I've done previously was use AWS ECS (Docker) with EFS (network storage) for persistence and RDS for the database. While this doesn't carry the same pricing model as Lambda, it is still cost efficient. You can set up your ECS Service to autoscale up and down. So that way you're running the bare minimum until you need more.
I've written a more in depth article about it here: https://serifandsemaphore.io/how-to-host-wordpress-like-a-boss-b5993fcfbd8e#.n6fbnf8ii ... but it's basically just the idea of running WordPress in Docker and using EFS to offload the persistent storage issues. You can swap many of the pieces of the puzzle out if you like. Use a database hosted in some other Docker service or Compose or where ever. That part need not be RDS for example. Even your storage could be handled in a different way, though EFS worked pretty well! The only major thing to note about EFS is the write speed. Most WordPress sites are read heavy though. Your mileage will vary depending on your needs.
Is it possible? Yes, anything is possible with enough time and effort. Is it worth it? That is a question best to ask yourself.
PHP can be run on Lambda as per the documentation located here: https://aws.amazon.com/blogs/compute/scripting-languages-for-aws-lambda-running-php-ruby-and-go/ .
The bigger initial problem as stated in other comments is a persistent file system. S3 for media storage is doable via Wordpress plugin (again from the comments) but any other persistent storage for the request / script execution is the initial biggest hurdle. Tackle one problem at a time till you get to the end!
I have an asp.net mvc 2 app, and I need to run a task (call WS, save into db) once a day. What is the suggested way for this? One thing, I have a feeling I would like to execute this task from within the webapp.
Does anyone have any suggestions:
I was considering .net quartz or regular System.Timers.Timer?
Does anyone see any problems?
Any other better solutions?
Thanks a lot
--MB
Why do you want to run this from the webapp? You have no way of even knowing if the webapp will be running at the required time. I would recommend you look at Windows Services
web apps are not the best host for a scheduled task. Unless you implement a system to keep the process awake, there's no chance to be sure your schedule will be executed.
Quartz.net is good for that.
It consists in a core module which you can use in your web app to save tasks and a server (windows service) module which executes your scheduled jobs.
I've recently implemented my own windows services and used Sql Server as storage for my scheduled jobs. Everything works pretty well, even if, I had to struggle a little bit to put things together.
The documentations is not always so clear.
Have a look at Quartz.Net, which is available as a NuGet package or from their site.
There are lots of examples of how to set these up, they are very flexible, you just have to define a class which implements IJob with a single Execute() method which gets fired by your choice of triggers.
You could also consider using Windows Workflow Foundation.
I need to create an ASP page (classic, not ASP.NET) which runs remote shell scripts on a UNIX server, then captures the output into variables in VBScript within the page itself.
I have never done ASP or VBScipt before. I have tried to google this stuff, but all I find are references to remote server side scripting, nothing concrete.
I could really use:
An elementary example of how this could be done.
Any other better alternatives to achieve this in a secure manner.
Are there any freeware/open source alternatives to these libraries?
Any examples?
If the shell scripts are normally run on a telnet session then you could screen scrape and parse the responses. There are commercial COM components out there such as the Dart telnet library: http://www.dart.com/pttel.aspx that would let you do this.
Either that or you could roll your own using AspSock http://www.15seconds.com/component/pg000300.htm
#Pascal, sadly I'm not aware of any F/OSS alternatives. We usually just buy in these types of libraries provided that they're not hugely expensive, and more often than not the cost is built into the customer's overall project cost.
If you had .NET on the server, you could build a COM wrapped component to do the heavy lifting around System.Net.Sockets.TcpClient. Just a thought.