How can I automate play store app creation - web-scraping

We have a template app written with flutter that each customer can buy a version of it and have the exact same functionality but connecting to a different database etc. Creating an app for each customer and deploying that app takes a lot of time, so we've decided to automate this process. For deploying updates of the apps, I was able to achieve that functionality very easily by integrating fastlane.
The part I am stuck at is how to automate app creation with all the steps (Creating the app, filling main store listing, publishing the first version, etc.). I've searched all over the internet and not found a single solution for this. For the App store there exists the fastlane produce tool, but according to the maintainers they do not plan to support an alternative for android at the moment.
As a solution I'm thinking of doing this using web scraping using tools such as puppeteer or selenium. Is there a better solution, and is doing scraping for this task a bad idea?

Related

How to avoid modifying Firebase Firestore data when working on new version of a app

I have developed an app (my first app) that's currently on app- and playstore. I'm using Firebase/Firestore as a back-end/database. I'm about to start to work on the next version of the app, which will require me to modify my database for testing and such. However, I currently have about 100 regular users on the app and I wouldn't want them to experience any weird data changes on their app while I'm developing.
I'm simply not sure about the way it's done when you further develop an existing app. I was thinking of creating new collections in Firestore simply for testing and hook my app with them but I don't know if that's the best way to go.
How do I work on a new version of an app without my users seeing any changes in their data?
If your data model is (very) simple you could create some specific, temporary, collections as you describe, but this is quite an error-prone approach (risk of modifying existing data, wrong or missing security rules, etc..) and is not recommended.
One standard approach is simply to create another Firebase project which is totally isolated from the production project. You will need to change the Firebase configuration in your Flutter app. If you need existing production data in your test Firebase project you could use the export/import mechanism.

Use different data for production and develop firebase sites

I have a CI/DC pipeline with google cloud build triggers that deploy my code to different sites depending on which branch I push to. The develop site is a live test - the final check before I merge to master, which triggers a deploy of master to the production site.
Currently, both sites use the same firebase Firestore db, and any document changed on the develop site will also be changed on the production site.
What I want to avoid is creating another firebase project to push the develop code to with a different database, because that means I need a separate set of credentials and would copy the same functions over to the new project every time I change them. That's not maintainable and is a lot of work.
What I would like is some way for the develop site to only have access to part of the firestore database, and the production site to have access to another part.
How do people do this? Is it even possible? Is there a better way? One alternative I can think of is using authentication and creating separate accounts for testing with different access permissions, but this seems a work-around and not the ideal solution.
What you're trying to do sounds like a lot more hassle than using multiple projects, which is the documented and strongly preferred solution. Putting everything in one project is a huge anti-pattern in Firebase and Google Cloud, and it will cause you more problems in the long run, in addition to increasing the risk of catastrophic failure if you manage to misconfigure something in that one project.
It's perfectly maintainable to have multiple projects like this, if you apply some scripting to automate the work. This is very common, and I strongly suggest thinking through how this would work for you.
You CI/CD pipeline could definitely check out your updates from source control and deploy them to whatever other project environments you have set up. It's very common to manage different credentials and configurations for use in CI/CD.

How to automate testing reporting dashboards

I am currently working on a project where we show users their metrics on dashboard and on reporting. QA team is testing it manually like running SQL queries in Database and comparing that number to UI of dashboard. Is there any way to automate this testing?
This application is written in .NET, C#. We dont use any specific reporting tool such as Cognos,....
If the dashboards are web based, you can use tools like QTP or Selenium which supports web application testing. You can also develop a Java based utility for the same if its just the numbers you want to compare. Basically, a framework which connects to the database, runs the reporting query at the backend and compares the numbers with the UI of the dashboard.
If there a way to export the dashboard or reports to excel then you can do an excel to excel comparison of the UI and backend database as well. This should be easier to achieve as compared to data comparison with backend DB.
We are using ARES dashboard(built under Testastra and owned by ZenQ) which is a good test automation dashboard based on just few api calls.
ARES, is an acronym for Test Automation Results dashboard. It's a TestAutomation framework/tool agonistic solution, that simplifies the collection of Test automation results and their analysis via live dashboard, daily/weekly trends, frequent failures etc.
Below are some of the features of ARES:
1. Tool agnostic test automation dashboard
2. SaaS based
3. Shows current live execution and historical test automation insights
4. Free for all
Website: http://www.testastra.com/#ares
Below repo has some code samples, documentation and usage of ARES test automation dashboard:
https://github.com/testastra/ARES
Worth trying.

Understanding the scalability of RShiny apps hosted on ShinyServer

I am building a series of interactive shiny web apps for a project that I am considering turning into a Company. My background is in data science and I don't have a lot of experience on the web app / server side of things, but these are important aspects for me to consider with my project. I currently have an Amazon Linux AMI EC2 instance with ShinyServer (free, open-source) installed, and I am currently hosting early versions of my web apps there. So far everything works fine, but I haven't made the links public yet.
My first question is whether anyone knows if there are certain limitations (scalability limitations, integration with database limitations, security / authentication limitations, etc.) that I will inevitably run into using RShiny apps and ShinyServer? I haven't heard of many successful, super-popular web apps being shiny apps hosted on ShinyServer, but rather my feeling is that ShinyServer is mainly used for hosting RShiny apps that are shared amongst only a small number of people (i.e. shared amongst team members at a company.). Per this thread - Does R-Server or Shiny Server create a new R process/instance for each user? - I am particularly concerned that my app won't be able to handle thousands of users simultaneously since only 1 R process is created for the app regardless of the # of concurrent users of the app. Having 10-20 processes through ShinyServer pro probably doesn't solve the issue either if I ever intend to scale greater than the hundreds or thousands of users. I also noticed that ShinyServer Pro would run me a not-so-negligible $10K per year.
My second question is whether RShiny apps can be deployed using other server technologies, such as Heroku. I came across this github page (https://github.com/virtualstaticvoid/heroku-buildpack-r/tree/heroku-16) but haven't dug too deep into it yet. I've been told that heroku makes it easy to update releases to apps whose code is on github (git push heroku:master), amongst other things.
My third question involves certain specific considerations of mine. In particular, I am currently working on a script that queries data from an API and writes that data to a (not-yet-setup) database of mine. This is the data my apps use, and I'd be interested in having the apps update in real time as the database updates, without requiring the user to refresh the webpage. A buddy of mine suggested AJAX for this type of asynchronous behavior, and it looks like this may be possible in R with something like this (https://github.com/daattali/advanced-shiny/tree/master/api-ajax).
Sorry that this is such a loaded question, but I hope it doesn't get closed down as I think it is fairly educational. Any suggestions / sources / pointing me in the right direction would be greatly appreciated on this.
Canovice,
I'd recommend you take a look at the following RStudio / AWS support articles. To scale a shiny server you'll need to look at using a load balancer:
RStudio
https://shiny.rstudio.com/articles/scaling-and-tuning.html
https://support.rstudio.com/hc/en-us/articles/220546267-Scaling-and-Performance-Tuning-Applications-in-Shiny-Server-Pro
https://support.rstudio.com/hc/en-us/articles/217801438-Can-I-load-balance-across-multiple-nodes-running-Shiny-Server-Pro-
AWS
https://aws.amazon.com/blogs/big-data/running-r-on-aws/
Blog Article:
http://mgritts.github.io/2016/07/08/shiny-aws/
Shiny is a great platform, their support is fabulous. I'd recommend you ring them up - they'll be sure to help answer your questions.
That said if your plan is to create a scalable website that will support thousands or hundreds of thousands of people then my sense would be to recommend you also review and consider using D3.js in conjunction with react.js or Angular.js, not forgetting to mention node.js.
My sense is that you are looking at a backend database connected to a logic engine and visualisation front end. If you are looking for a good overview of usage take a look at the following web page and git repo [A little dated but useful]:
https://anmolkoul.wordpress.com/2015/06/05/interactive-data-visualization-using-d3-js-dc-js-nodejs-and-mongodb/
https://github.com/anmolkoul/node-dc-mongo
I hope the above points you in the right direction.
I'd like to provide some notes related to your second question: Yes, you can use the mentioned buildback to deploy shiny applications on heroku.
I was in a similar situation with you (asking myself about possible ways of serving Shiny applications in a scalable manner) and decided to go the "heroku way".
You may find these hints helpful when deploying your app to heroku using the buildpack mentioned above:
Heroku tries to "guess" how to execute your application. But you can also add a special file, named Procfile, to your application to control the process commands you want to execute for your application. In my case I used web: R -f ~/run.R --gui-none --no-save, where this means that a file named run.R is being passed to the R executable for the web server process
The stack on heroku is based on Ubuntu. If you need additional deb-packages, you can create another special file named Aptfile and add the package names therein, heroku will then automatically install these for you (I needed it for RPostgreSQL)
You can add another special file named init.R and install all R packages as necessary just as you are used to, i.e. with install.packages etc. You can also add initial configuration material within this file.
As a running example, here is an example toy application that I wrote for myself to remember how a "full-stack" shiny app may look like, including compability with heroku.
For a large number of concurrent users, use a load balancer like nginx and enable the autoscaling of your app, e.g. through Kubernetes.
You can deploy your app on Heroku. On the paid tiers it includes NoOps autoscaling of your app. See this tutorial on how to deploy a Shiny app in a Docker container on Heroku: https://medium.com/analytics-vidhya/deploying-an-r-shiny-app-on-heroku-free-tier-b31003858b68
You can query the table last update timestamp in the Shiny server logic with reactivePoll() and rerun your db query if it changed. It is not "real-time" but depending on your application close enough if you set the time interval small.

Download an app from the app store, without the UI

I'm working on a tweak that needs to download and install an app from the App Store (given a bundle id or some kind of other identifier), but doing so without using the App Store app (e.g., to allow the user to remotely instal an app on the device, without going through the App Store UI).
(I'll pause here to point that obviously this has nothing to do with piracy, as I want the app to be downloaded through the user's account in the store, and in any case I'm only targeting free apps at the moment).
I went through iOS 7 private framework headers and couldn't find a way to fit in and to tell the device to install an app with a given identifier. I've also looked for similar tweaks to have a look at their source, but the only one I could find was 'Auto App Updater', and its code isn't public.
I would greatly appreciate your help in coming up with a code that does that, or an open-source tweak that utilises a similar function.
Thanks a lot!
Dan

Resources