I am working on a Cloud Project.
I want to modify the Eucalyptus code for load balancing in the cloud. I have the source code configured with Eclipse. I want to know the dependent files of load balancer in the source code and I need to change them to add a dynamic load balancer in Eucalyptus. Can anyone help me out?
It is my understanding the ELB is actively discussed and should appear in Eucalyptus soon. TO the original question, if you want to write your own version of ELB, you will need to modify Eucalyptus extensively, starting from the API (do you want to use AWS version? Implement your own one?), going to get the real work done. It may be easier to hang out on #eucalyptus-devel (freenode) and start asking questions there.
Related
I'm not a server guy, but for my website I have started learning about aws because I have migrated my website from godaddy to aws.
I have searched for my problem in many tutorials, documentation, youtube and stackoverflow as well but could not find it. I'm struggling with this for 1 week.
I have installed wordpress on my aws using bitnami. My website was running with 1 instance i.e free created using bitnami and I have added more than 22k records on my wordpress site so My site started loading very slow because of heavy data in the database which was causing 100% cpu usages so often then I did the following
added load balancer and attached existing instance with elb
created image of existing instance (bitnami wordpress instance)
created the launch configuration under auto scaling
created auto scaling group
now 1 more instance started showing
I have added this new instance to load balancer as well
now website is loading very fast because the load is divided into 2 servers
Now the prob is
both servers have the different database I need to add more data, how I can add data? do I need to add data to both server database individually?
As both servers are different so both have different files and I need to make lot of changes in the files because website is under development. do I need to make changes in both server files always?
is there a way that both servers serve the same database and same files so I don't need to make changes in 2 different places everytime
I will be really thankful if anyone can get me out of here. I believe this is a common problem many people should have faced but I did not find anything like this anywhere which could help me.
Thanks,
You need to take your database out of the Auto Scaling Group and put other materials in a shared storage. You could use Amazon RDS as your database and Amazon S3 as a shared storage. Both of these services are scalable so you don't need to worry about scaling in/out in the future. Also refer to this tutorial for deploying a high availability WordPress website.
I got it resolved by moving to RDS and it works like a charm. now my files and database are hosting in different servers and website loading speed also improved.
I have tested website with some load testing tools and it can handle some huge number of traffic now.
Note: I have shut down the old instance now website is working on 1 instance only and its working beter than before.
Thanks #mahdi your answer helped me.
I am building a series of interactive shiny web apps for a project that I am considering turning into a Company. My background is in data science and I don't have a lot of experience on the web app / server side of things, but these are important aspects for me to consider with my project. I currently have an Amazon Linux AMI EC2 instance with ShinyServer (free, open-source) installed, and I am currently hosting early versions of my web apps there. So far everything works fine, but I haven't made the links public yet.
My first question is whether anyone knows if there are certain limitations (scalability limitations, integration with database limitations, security / authentication limitations, etc.) that I will inevitably run into using RShiny apps and ShinyServer? I haven't heard of many successful, super-popular web apps being shiny apps hosted on ShinyServer, but rather my feeling is that ShinyServer is mainly used for hosting RShiny apps that are shared amongst only a small number of people (i.e. shared amongst team members at a company.). Per this thread - Does R-Server or Shiny Server create a new R process/instance for each user? - I am particularly concerned that my app won't be able to handle thousands of users simultaneously since only 1 R process is created for the app regardless of the # of concurrent users of the app. Having 10-20 processes through ShinyServer pro probably doesn't solve the issue either if I ever intend to scale greater than the hundreds or thousands of users. I also noticed that ShinyServer Pro would run me a not-so-negligible $10K per year.
My second question is whether RShiny apps can be deployed using other server technologies, such as Heroku. I came across this github page (https://github.com/virtualstaticvoid/heroku-buildpack-r/tree/heroku-16) but haven't dug too deep into it yet. I've been told that heroku makes it easy to update releases to apps whose code is on github (git push heroku:master), amongst other things.
My third question involves certain specific considerations of mine. In particular, I am currently working on a script that queries data from an API and writes that data to a (not-yet-setup) database of mine. This is the data my apps use, and I'd be interested in having the apps update in real time as the database updates, without requiring the user to refresh the webpage. A buddy of mine suggested AJAX for this type of asynchronous behavior, and it looks like this may be possible in R with something like this (https://github.com/daattali/advanced-shiny/tree/master/api-ajax).
Sorry that this is such a loaded question, but I hope it doesn't get closed down as I think it is fairly educational. Any suggestions / sources / pointing me in the right direction would be greatly appreciated on this.
Canovice,
I'd recommend you take a look at the following RStudio / AWS support articles. To scale a shiny server you'll need to look at using a load balancer:
RStudio
https://shiny.rstudio.com/articles/scaling-and-tuning.html
https://support.rstudio.com/hc/en-us/articles/220546267-Scaling-and-Performance-Tuning-Applications-in-Shiny-Server-Pro
https://support.rstudio.com/hc/en-us/articles/217801438-Can-I-load-balance-across-multiple-nodes-running-Shiny-Server-Pro-
AWS
https://aws.amazon.com/blogs/big-data/running-r-on-aws/
Blog Article:
http://mgritts.github.io/2016/07/08/shiny-aws/
Shiny is a great platform, their support is fabulous. I'd recommend you ring them up - they'll be sure to help answer your questions.
That said if your plan is to create a scalable website that will support thousands or hundreds of thousands of people then my sense would be to recommend you also review and consider using D3.js in conjunction with react.js or Angular.js, not forgetting to mention node.js.
My sense is that you are looking at a backend database connected to a logic engine and visualisation front end. If you are looking for a good overview of usage take a look at the following web page and git repo [A little dated but useful]:
https://anmolkoul.wordpress.com/2015/06/05/interactive-data-visualization-using-d3-js-dc-js-nodejs-and-mongodb/
https://github.com/anmolkoul/node-dc-mongo
I hope the above points you in the right direction.
I'd like to provide some notes related to your second question: Yes, you can use the mentioned buildback to deploy shiny applications on heroku.
I was in a similar situation with you (asking myself about possible ways of serving Shiny applications in a scalable manner) and decided to go the "heroku way".
You may find these hints helpful when deploying your app to heroku using the buildpack mentioned above:
Heroku tries to "guess" how to execute your application. But you can also add a special file, named Procfile, to your application to control the process commands you want to execute for your application. In my case I used web: R -f ~/run.R --gui-none --no-save, where this means that a file named run.R is being passed to the R executable for the web server process
The stack on heroku is based on Ubuntu. If you need additional deb-packages, you can create another special file named Aptfile and add the package names therein, heroku will then automatically install these for you (I needed it for RPostgreSQL)
You can add another special file named init.R and install all R packages as necessary just as you are used to, i.e. with install.packages etc. You can also add initial configuration material within this file.
As a running example, here is an example toy application that I wrote for myself to remember how a "full-stack" shiny app may look like, including compability with heroku.
For a large number of concurrent users, use a load balancer like nginx and enable the autoscaling of your app, e.g. through Kubernetes.
You can deploy your app on Heroku. On the paid tiers it includes NoOps autoscaling of your app. See this tutorial on how to deploy a Shiny app in a Docker container on Heroku: https://medium.com/analytics-vidhya/deploying-an-r-shiny-app-on-heroku-free-tier-b31003858b68
You can query the table last update timestamp in the Shiny server logic with reactivePoll() and rerun your db query if it changed. It is not "real-time" but depending on your application close enough if you set the time interval small.
When doing web development there are times external resources are referenced in a web page (e.g. google fonts). I would like to cache some of these calls on my Macbook but not cache the code Im working on.
The goal being speed of development and a workaround when working on slow networks (e.g. 3G using a hotspot).
I came across Squid proxy but have not been able to configure it at all. Im open to other suggestions to achieve this goal. Any ideas?
I'm using SquidMan : http://squidman.net
It come with a nice user interface.
I would like to develop a website for a company with a lot of different functions. I decided to use the Symfony 2.0 framework for the first time and started reading the documentation and also a video tutorial.
After a week reading the documentation I still have some general questions unanswered. It would be great if you could help me with them:
Which client OS do you recommend to work with Symfony2? Windows doesn't seam appropriate.
Can I develop the whole website on a local Apache server and than upload the files to a regular webhosting server and use it?
I bought a regular web hosting (www.cyon.ch) is this recommended? I can't use any console on it.
I'm really confused and I'm sure I'm asking quite some stupid questions. It would be great if you can help me understand better how Symfony2 works.
Thanks in advance!
Windows is OK, as long as you have PHP and MySQL. There are several package that can do the job.
You can. In fact you should. Common workflow is develop on localhost and test on test server, and then move to production server. You should use a SCM like git.
Theoretically you can. But you won't have access to many console features that are really useful, like updating the database schema, flushing cache, download dependecies automatically, etc. I know some people doing this (copying everything with vendors), that's heavy and insecure.
When it takes half a day to update the app, you don't hurry when you have to apply security patch. Unlike Wordpress or Drupal, Symfony has no other 'easy way' to update vendors and third party modules than command line interface. So it is really not a good idea. You can find a lot of VPS with full ssh access, that cost less than 10$ months.
I work on windows just fine. For wamp server, use Easyphp. Everything is easy to configure and most important; cli and apache users use same php.ini
Yes, but it is bad. You should use either svn or git. I find svn easier to use and both of them have UI variants: TortoiseSVN and TortoiseGIT.
No. You need console access and php5.3 minimum.
I am fairly new to dealing with high traffic websites and I'm looking for a setup that will be able to scale well into the future.
My site currently uses one main script to pull data and store it into a database. The rest of the website is mostly presentation of this data from the database.
I have been looking at the possibility of using Amazon Web Services (EC2) or Google Compute Engine for running the main script. Then transferring that data to a more typical web hosting service. One thing I am not sure that I need to worry about is load balancing, I've seen that using NGINX for the load balancing can help improve the performance.
My goal is to have the site up 99.99% of the time and execute the script as fast as possible. It would be optimally nice to have a great support in case something goes wrong too.
Any ideas, comments, or suggestions are welcome. I'm am trying to learn how to handle this as I have a couple other websites that are currently growing and may need such a setup in the near future.
Thanks!
Google Compute Engine is still in its very early stages. For that reason alone, I would recommend using EC2. With EC2, you have all of the features you need. The ELB can seamlessly load balance between any amount of servers, which would help with the 99.99% uptime.
Using CloudFormation, you can define templates for scaling to run that script in a clustered environment.
Hope this helps.
Just find yourself a good host with a decent SLA, running a relational database within the cloud is asking for problems when you do get high database loads.
Don't be fooled, the cloud is a really nice solution for some problem.. but NOT ALL!
Same goes for NoSQL, and the combination of cloud and NoSQL.
Do your own research and don't be hoaxed by the marketing machines and the people not doing their homework.
Loadbalancing is nice but big change your database is the first bottleneck you run into.
Don't optimize thing before you need to!
If you are asking me, find yourself a decent dedicated (non-virtual) server with a decent SLA and backup plan.
Calculate howmuch 99.9% uptime actually is and realize that there is no problem in reaching that aslong as you have a decent SLA and proper backups.
You can always go from iron to the cloud,.. the other way around is a different story.