I currently manage Web App on a LAMP stack hosted with GreenGeeks. As it has scaled up, I have started learning MeteorJS on my local machine and am thinking about redeveloping the app in Meteor to support more concurrent connections. My questions are:
Can Meteor simply be hosted in a Simple Amazon S3 Bucket with no need for a stack of any kind? Is this smart? When something seems this simple, I get nervous.
Is Meteor as portable as it feels? Migrating a LAMP app from one server to another can be a real pain. This "feels" like it's as simple as zipping up the whole thing and simply dragging it anywhere. Again, feels too simple = nervous.
Is meteor the right choice if I am looking to maximize concurrent connections and reduce the number of times I need to go to the server for information? My app loads about 2 MB of data per user and I'd love a situation where this can be loaded once and the user has it available to interact with without going to the server (unless it changes).
Ok answers to your questions:
Well actually You can deploy your meteor app into an Amazon EC2 instance, the process is pretty easy, take a look to This video
Meteor is incredible portable, actually it was made with nodejs, therefore it inherits its features
You are in the right way, you know meteor is reactive, it acts in real time, also uses mongoDB, which is incredible faster than a regular SQL database, so in general meteor's performance is amazing, in fact, there are lots of packages that improve even more the performance of your app like this one and many others
Related
I am currently thinking about the best way to deploy my RShiny app. After trying to host my app on a dedicated server via Shinyproxy, Docker and Nginx - but this solution was (surprise!) not really scalable. The RAM requirement per user was too high for that.
I'm currently considering hosting the app via a Docker image in AWS Fargate, where RAM resources scale up and down as needed.
I'm now wondering about security, though.
Brief background:
My goal is to add my app as a tool to an online store. Here it can and will (hopefully) happen that several users will use the tool at the same time. It's important that users can't mess with each other's data - that's why I thought of ShinyProxy, so that each user gets their "own R session".
Now I am wondering what this looks like with AWS Fargate. Could it be that if multiple users are active in the tool at the same time, there can be mutual interference?
If so, does anyone have any ideas on how to prevent this? Unfortunately, publishing ShinyProxy via Fargate is not possible as far as I know.
I hope I could formulate my question understandably and someone of you can help me.
Thank you and have a nice day!
Brief background: My goal is to add my app as a tool to an online
store. Here it can and will (hopefully) happen that several users will
use the tool at the same time. It's important that users can't mess
with each other's data - that's why I thought of ShinyProxy, so that
each user gets their "own R session".
Probably depends on what you need for your use case.
Shiny actually has no user management per default - in the sense of limiting access to your application for certain groups and requiring authentication (can be done by hosting with Shinyapps.io and others).
But you probably do not really need this anyway - your problem sounds more like a scoping issue.
(you should read this information about it)
Sure, there might only be one R process, but it actually supports multiple client connections (sessions). You can define, what objects these sessions share. This is totally independent from where you host your app.
Everything you put into the shinyServer() function in the server.R file will only be visible within the user session. (every user has it's own session)
If you need to share variables between sessions, you have to put them in the server.R file, but outside of the shinyServer() function.
I am building a series of interactive shiny web apps for a project that I am considering turning into a Company. My background is in data science and I don't have a lot of experience on the web app / server side of things, but these are important aspects for me to consider with my project. I currently have an Amazon Linux AMI EC2 instance with ShinyServer (free, open-source) installed, and I am currently hosting early versions of my web apps there. So far everything works fine, but I haven't made the links public yet.
My first question is whether anyone knows if there are certain limitations (scalability limitations, integration with database limitations, security / authentication limitations, etc.) that I will inevitably run into using RShiny apps and ShinyServer? I haven't heard of many successful, super-popular web apps being shiny apps hosted on ShinyServer, but rather my feeling is that ShinyServer is mainly used for hosting RShiny apps that are shared amongst only a small number of people (i.e. shared amongst team members at a company.). Per this thread - Does R-Server or Shiny Server create a new R process/instance for each user? - I am particularly concerned that my app won't be able to handle thousands of users simultaneously since only 1 R process is created for the app regardless of the # of concurrent users of the app. Having 10-20 processes through ShinyServer pro probably doesn't solve the issue either if I ever intend to scale greater than the hundreds or thousands of users. I also noticed that ShinyServer Pro would run me a not-so-negligible $10K per year.
My second question is whether RShiny apps can be deployed using other server technologies, such as Heroku. I came across this github page (https://github.com/virtualstaticvoid/heroku-buildpack-r/tree/heroku-16) but haven't dug too deep into it yet. I've been told that heroku makes it easy to update releases to apps whose code is on github (git push heroku:master), amongst other things.
My third question involves certain specific considerations of mine. In particular, I am currently working on a script that queries data from an API and writes that data to a (not-yet-setup) database of mine. This is the data my apps use, and I'd be interested in having the apps update in real time as the database updates, without requiring the user to refresh the webpage. A buddy of mine suggested AJAX for this type of asynchronous behavior, and it looks like this may be possible in R with something like this (https://github.com/daattali/advanced-shiny/tree/master/api-ajax).
Sorry that this is such a loaded question, but I hope it doesn't get closed down as I think it is fairly educational. Any suggestions / sources / pointing me in the right direction would be greatly appreciated on this.
Canovice,
I'd recommend you take a look at the following RStudio / AWS support articles. To scale a shiny server you'll need to look at using a load balancer:
RStudio
https://shiny.rstudio.com/articles/scaling-and-tuning.html
https://support.rstudio.com/hc/en-us/articles/220546267-Scaling-and-Performance-Tuning-Applications-in-Shiny-Server-Pro
https://support.rstudio.com/hc/en-us/articles/217801438-Can-I-load-balance-across-multiple-nodes-running-Shiny-Server-Pro-
AWS
https://aws.amazon.com/blogs/big-data/running-r-on-aws/
Blog Article:
http://mgritts.github.io/2016/07/08/shiny-aws/
Shiny is a great platform, their support is fabulous. I'd recommend you ring them up - they'll be sure to help answer your questions.
That said if your plan is to create a scalable website that will support thousands or hundreds of thousands of people then my sense would be to recommend you also review and consider using D3.js in conjunction with react.js or Angular.js, not forgetting to mention node.js.
My sense is that you are looking at a backend database connected to a logic engine and visualisation front end. If you are looking for a good overview of usage take a look at the following web page and git repo [A little dated but useful]:
https://anmolkoul.wordpress.com/2015/06/05/interactive-data-visualization-using-d3-js-dc-js-nodejs-and-mongodb/
https://github.com/anmolkoul/node-dc-mongo
I hope the above points you in the right direction.
I'd like to provide some notes related to your second question: Yes, you can use the mentioned buildback to deploy shiny applications on heroku.
I was in a similar situation with you (asking myself about possible ways of serving Shiny applications in a scalable manner) and decided to go the "heroku way".
You may find these hints helpful when deploying your app to heroku using the buildpack mentioned above:
Heroku tries to "guess" how to execute your application. But you can also add a special file, named Procfile, to your application to control the process commands you want to execute for your application. In my case I used web: R -f ~/run.R --gui-none --no-save, where this means that a file named run.R is being passed to the R executable for the web server process
The stack on heroku is based on Ubuntu. If you need additional deb-packages, you can create another special file named Aptfile and add the package names therein, heroku will then automatically install these for you (I needed it for RPostgreSQL)
You can add another special file named init.R and install all R packages as necessary just as you are used to, i.e. with install.packages etc. You can also add initial configuration material within this file.
As a running example, here is an example toy application that I wrote for myself to remember how a "full-stack" shiny app may look like, including compability with heroku.
For a large number of concurrent users, use a load balancer like nginx and enable the autoscaling of your app, e.g. through Kubernetes.
You can deploy your app on Heroku. On the paid tiers it includes NoOps autoscaling of your app. See this tutorial on how to deploy a Shiny app in a Docker container on Heroku: https://medium.com/analytics-vidhya/deploying-an-r-shiny-app-on-heroku-free-tier-b31003858b68
You can query the table last update timestamp in the Shiny server logic with reactivePoll() and rerun your db query if it changed. It is not "real-time" but depending on your application close enough if you set the time interval small.
As I heard that there are many things need to be considered before Deploying a Meteor App, However, it's still quite vague. Anyone please give me some opinions about this issue. Thanks
This is probably the wrong forum for this type of question, since it's highly opinionated and not really in the Q&A format, but I'll give you some of my personal opinions.
Where am I hosting my app?
This has a lot to do with what your app is (Web-based app, Android-only), how many users you plan to have, if this is a public app or something private, how much you have to spend, and many other factors. Options include:
Host it yourself - Could be on a VPS (Virtual Private Server, like Digital Ocean and others), some cloud offering (AWS or similar), or a bare-metal server you have hosted somewhere (like in your closet).
Pay for dedicated hosting - Several out there that offer many different features, like Galaxy or Modulus, etc.
If you host it yourself, then you have to maintain the hosting solution, I.E you need to support it all on your own. This may mean provisioning/installing the OS, installing and configuring the server apps (MongoDB, Node.js, web servers, etc), and maintaining everything over time. The benefits, however, are potentially cheaper hosting costs (although this could be debated) and custom setups/architectures. If you are creating an app that should remain private (I.E. not a public app) you may want to consider this option so you can host it internal to the company and not make it public-facing. There are some tools out there that can ease the process of setting up the server, for instance MUP/MUPX.
For dedicated Meteor hosting, the benefit is that someone else does the installation, configuration, and maintenance of the core apps, and all you have to do is push the button to move your code in. These options can be more pricey as they are covering the IT costs of supporting the environment, but they usually come with the benefit that a) You don't have to install all that stuff yourself, b) you don't have to be an expert at all the "plumbing", c) You don't have to hire a staff of people to support your infrastructure, d) These hosting services usually know how to optimize things properly to get better performance for your app.
Do I deploy manually or do I use some tool?
This depends highly on the answer to #1, as your hosting decision may come with it's own set of tools used to deploy (ex. Galaxy), or you may need to shop around for the best tool for you. For manual deploys, I'd suggest looking at MUP/MUPX, which can automate the deployment of your app, even configuring the web server, DB server, and setting up everything as Docker images. Or, if you want to have more control, maybe take a look at something like Grunt or Gulp, which are more build-scripting solutions (similar to ANT/Maven/Gradle are for Java).
Do I expect a fast growth, or a slow trickle?
Again, this has a lot to do with where you plan to host. Lots of cloud services make it easy to grow/shrink your cluster of servers based on load, but this takes a LOT of configuration past just installing an OS. VPS and bare-metal solutions will be the hardest to expand. Dedicated hosting will depend on the provider.
You need to serious need to think about how you might handle a fast-growth situation, even if you don't think your app will take off. The internet is riddled with apps that failed because they didn't think they would be as successful as they were. It only takes one mention on something like Reddit, Y Combinator, or Product Hunt for your app to get a sudden and unexpected rush of traffic that takes down the server(s). If you know your growth will be controlled in some way, like if you have a private app with a pre-set number of users, then you might not need to worry about it.
Do I need to monitor my app?
The answer to this is always "Yes", but to what extent? Do you need to provide 24x7 uptime? Would an outage cost you lots of money, make you loose your biggest client, etc? Can the users live without service for a little while if it goes down, or would I loose face and customers? Depending on how serious you need to be, you should consider some sort of monitoring of your infrastructure and app. Again, there are LOTS of options here, and your decision might be swayed quite a bit by the answer to the first question.
I am sure there are other questions, but these are the biggest I could come up with.
I am wondering if it is feasible to deploy wordpress as a series of lambda functions on AWS API gateway. Any pointers on the feasibility/gotchas would be greatly appreciated!
Thanks in advance,
PKK
You'll have a lot of things to consider with persistence and even before that, Lambda doesn't support PHP. I'd probably look at Microsoft Azure Functions instead that do support PHP and do have persistent storage.
While other languages (such as Go, Rust, Swift etc.) can be "wrapped" to run in AWS Lambda with relative ease, compiling PHP targeting the same platform and running it is a bit different (and certainly more painstaking). Think about all the various PHP modules you'd need for starters. Moreover, I can't imagine performance will be as good as something like a Go binary.
If you can do something clever with the Phalcon framework and come up with an easy build and deploy process, then maayyyybee.
Though, you'd probably need to really overhaul something like WordPress which was not designed for this at all. It still uses some pretty old conventions due to the age of the project and while that is all well and good for your typical PHP server, it's a different ball game in the sense of this "portable" PHP installation.
Keep in mind that PHP sessions are relied upon as well and so you're going to need to move those elsewhere due to the lack of persistence with AWS Lambda. You can probably find some sort of plugin for WordPress that works with Redis?? I have to imagine something like that has been built by now... But there will be many complications.
I would seriously consider using Azure Functions to begin with OR using Docker and forgoing the pricing model that cloud functions offers. You can still find some pretty cheap and scalable hosting out there.
What I've done previously was use AWS ECS (Docker) with EFS (network storage) for persistence and RDS for the database. While this doesn't carry the same pricing model as Lambda, it is still cost efficient. You can set up your ECS Service to autoscale up and down. So that way you're running the bare minimum until you need more.
I've written a more in depth article about it here: https://serifandsemaphore.io/how-to-host-wordpress-like-a-boss-b5993fcfbd8e#.n6fbnf8ii ... but it's basically just the idea of running WordPress in Docker and using EFS to offload the persistent storage issues. You can swap many of the pieces of the puzzle out if you like. Use a database hosted in some other Docker service or Compose or where ever. That part need not be RDS for example. Even your storage could be handled in a different way, though EFS worked pretty well! The only major thing to note about EFS is the write speed. Most WordPress sites are read heavy though. Your mileage will vary depending on your needs.
Is it possible? Yes, anything is possible with enough time and effort. Is it worth it? That is a question best to ask yourself.
PHP can be run on Lambda as per the documentation located here: https://aws.amazon.com/blogs/compute/scripting-languages-for-aws-lambda-running-php-ruby-and-go/ .
The bigger initial problem as stated in other comments is a persistent file system. S3 for media storage is doable via Wordpress plugin (again from the comments) but any other persistent storage for the request / script execution is the initial biggest hurdle. Tackle one problem at a time till you get to the end!
I’m using EE for the first time. I design/develop on two machines, so I use private github repos for all my client work. I’d like to run this EE project on SQLite locally while I’m developing and designing, and then switch it to MySQL once I’m ready to deploy to the staging environment for client review. That way, I can track DB changes easily between machines, as there is a lot of content on this particular site, and exporting/importing DB tables every time I switch machines (three or more times a day) is getting old.
I realize I could use the client’s MySQL DB remotely, but would rather keep everything local, if at all possible. So, how can I run EE with SQLite?
This question (or something similar) was made almost three years ago here. The accepted answer hints at possible support in EE v2.0+ (via CodeIgniter’s then-new database abstraction), but I can’t find any recent information (official or otherwise) on how to make it happen.
I short: no. Active Record is not universally-used throughout EE, and especially not in third party add-ons. In fact, the documentation for the Database class does not mention Active Record at all.
I'd suggest just installing MySQL locally.
I don't think you'll have much luck with non-MySQL DBs — MySQL statements are hard-coded into EE, despite the EE 2.0 promises of a fully-fledged CI app with various DB connectors.
I think your best results might be to find a centralized host for your dev DB. EE is especially painful to work with on multiple machines because of the DB — the whole thing is really built with "one server, one developer" as the primary use case.
You can simplify it a bit by keeping your templates saved as files, and rigging your settings to be server-agnostic. I also keep little scripts around to quickly dump/import the DB when I have to, which makes moving the site around in a git repo a little easier. A few details here: http://www.viget.com/inspire/expressionengine-on-multiple-machines/