My question sound creazy :
is there a way to create an AWS Glue Databrew and then include it in an AWS Glue Studio ?
Databrew is easy to use, friendly UI, no code, a lot of functions.
Glue studio, UI too, but in the end you have always something to code.
Since ETL include 'transform' how could I benefit of Glue Studio power and all the UI/functions available to preprocess files from Databrew ?
Thank you for any suggestion :)
I tried to Google, and I explored AWS GLue Studio and Databrew
Related
Can I use R scripts in USQL while working on local microsoft studio? All documents that i can find are related to Azure execution only.-
https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/data-lake-analytics/data-lake-analytics-u-sql-r-extensions.md
Till last year, this was not done. Any Idea if we can use R scripts in Visual Studio( USQL project) now ?
https://social.msdn.microsoft.com/Forums/en-US/938d423f-b163-4273-87af-cc0af7b301ec/r-in-adla-script?forum=AzureDataLake
Yes, please see Enabling U-SQL Advanced Analytics for Local Execution.
I am storing my data in S3 bucket provided from Amazon. I am performing analytics in R studio by creating instance.
I am storing my desired result back in my S3 bucket.
I would like to create an front end in visual studio for my results in R. Could anyone guide me on how I could proceed with this ?
I have created the AWS explorer in visual studio.
Any lead and guidance for this idea would be helpful.
But seriously, take a look at shiny
Plagiarizing from that link: "Shiny is an R package that makes it easy to build interactive web apps straight from R. You can host standalone apps on a webpage or embed them in R Markdown documents or build dashboards. You can also extend your Shiny apps with CSS themes, htmlwidgets, and JavaScript actions."
Options (not in any order):
Shiny is an R package that makes it easy to build interactive web apps straight from R
If front-end is all about reports/dashboard, you can push your results in ElasticSearch and then use Kibana on top to present the results as a dashboard.
HIH
I am using RStudio. Installed a local version of Spark, run a few things, quite happy. Now I am trying to read my actual data from a Cluster, using RStudio Server and a standalone version of Spark. Data is in Cassandra, and I do not know how to connect to it. Can anyone point me to a good primer on how to connect and read that data?
there is a package called crassy and is listed on the official sparklyr site's under Extensions. You can search for crassy in GitHub.
I have an R-script that does stuff with a bunch of tweets and I would like to use the same script on the same data but saved in an Hadoop file system. According to this Hortonworks tutorial I could use R code with data from my HDFS, but it is not quite clear.
Can I use the very same R-script, taking advantage of the mapreduce paradigm, by using this Revolution R? Should I change my code or is there a way to execute the same functions optimized for an Hadoop architecture?
My wish would be to write my code on a standard R IDE like R-Studio and then use it, or use the most of it, on my cloud services (such as Microsoft Azure) with mapreduce on the base.
Yes, you can run any R script across different data platform from Hadoop to Spark to Teradata and SQL Server by using environment specific compute context.
Following two links should help you get started on how to use Revolution R / Microsoft R Server on Hadoop:
https://msdn.microsoft.com/en-us/microsoft-r/scaler-hadoop-getting-started
https://github.com/Azure/Azure-MachineLearning-DataScience/blob/master/Misc/MicrosoftR/Samples/NYCTaxi/NYC2013_MRS_LinearBinary.Rmd
I am interested in testing flyway and if I am not wrong I read that it supports db changes both through java and SQL. I am a dba and familiar with SQL but not java.
I read through the “Getting Started” page and wanted to try out the sample application available under the “Downloads tab” link however I couldn’t find any readme document explaining the available downloads which appeared to contain .jar files.
Q) is there an instruction manual for a newbies to explain how to put together this sample application?
Q) can one uses flyway without knowing java? If yes, please provide any how-to url/notes/documents available. If not do you have any how-to for one to get started with java just enough to operate this tool?
Thanks Bob
I think you might find the command line tool useful:
http://flywaydb.org/documentation/commandline/
As it says on the website:
The Flyway command-line tool is meant for users who
do not run their applications on the JVM
wish to migration their database from the command-line without having to install Maven
You may need to browse the source code to figure out some more details:
https://github.com/flyway/flyway
Although I think you should be able to adapt the regular documentation to the CLI option.
Try starting here:
http://flywaydb.org/getstarted/existingDatabaseSetup.html