I have several functions on the dev ADX environment. Now I need to migrate all of them into upper environment. Is there any handy commands to export all available functions from one environment to another. I tried to export all functions as csv after running .show functions command. But could not find any way to import this csv file to another environment.
The following command will produce you a csl script that will recreate all your tables, functions etc.: .show database schema as csl script - just go over its output and take only what you need.
Related
I am trying to automate my slackr posts through Github Actions and I have a question:
Was trying to run an R script that included slackr through a github aciton and I keep running into a problem where my script runs without any errors but fails to post to slack because while the requisite environment variables ('SLACK_CHANNEL', 'SLACK_TOKEN', 'SLACK_USERNAME', etc.) are present in the action environment, they're all unfortunately blank.
It was my understanding that I could set up environment variables ahead of time through Actions Secrets, call the env and specify the variables in my yaml script, and simply use slackr() along with Sys.getenv() to point the script to my env variables rather than using slackr_setup and exposing my API key.
While I'm encouraged by the fact that R seems to be seeing the env variables I set up through Action secrets, I am still unsure about why R thinks they are blank. Could it be that I am declaring the environment at the wrong point in the yaml file? Is R simply incompatible with Actions Secrets? Something else?
empty env variables
It is possible to run a R script with Pentaho, but instead of export the result as a csv file, insert the result directly into a table on a DB?
Using the Community Edition of Pentaho, you could use a script executor step to execute a shell script in your OS to do all the work, including inserting to the database, which is not much Pentaho related, all the work is done by the shell script and you just use Pentaho to call the execution of that script.
There's also a very old plugin available in Github that I don't know if it would work with modern versions of Pentaho and R, to execute R code within Pentaho and then continue the stream of data to "normal" steps like the table output to insert the data to a table.
These are the details to configure that plugin from the developers:
http://dekarlab.de/wp/?p=5
I explored the log4r package and found that we have console and file appenders. I am looking for pushing the logs to a mongoDB database. I am unable to find any way in pushing the logs to mongoDB, do we have any appender for the same.
TIA
You can try the mongoimport tool
If you do not specify a file (option --file=), mongoimport reads data from standard input (e.g. "stdin").
Or write logs into a temporary file and use mongoimport to import it.
Kusto (ADX) has .export command to export data from Kusto to other data stores (Azure Storage is of my particular interest though). Here I am not referring to continuous export but rather just export command which can also be used in on-demand manner. I think couple of years back this was the only (out of the box) option to export data out of Kusto. Now I see that ADFv2 (Azure Data Factory v2) copy activity also supports ADX (Kusto) as one of the sources and using this we are able to copy data from Kusto to Azure Storage. But which method is faster ? Is there any recommendation? One could also orchestrate .export command from ADFv2's ADX command activity which is what we were doing earlier -- but now I see that copy activity from ADFv2 has started supporting ADX as source out of the box , which is much easier to work with. But we only want to use the method which will provide best performance (faster).
Please see the comparison between copy activity and the export command here specifically the tip here:
The reason for it, is that the ".export" command perform better as it by default executed in parallel as well as providing more customization options.
I'm trying to build an R package whose goal is to run a series of analyses by taking input data and writing output data to an external database (PostgreSQL).
Specifically, I need a set of operations to be scheduled to run on a daily basis. Therefore, I have written some bash scripts with R code (using the header #!/usr/bin/env Rscript) and I have saved them into the exec/ folder of the R package. The scripts make several call to the package's core functions in R/ folder.
At this point, once installed the package on a linux server, how do I set up a crontab that is able to directly access the scripts in the exec/ folder?
Is this way of proceeding correct or is there a different best practice for such operations?
We do this all the bleeping time at work. Here at home I also have a couple of recurring cronjobs, eg for CRANberries. The exec/ folder you reference works, but my preferred solution is to use, say, inst/scripts/someScript.R.
Then one initial time you need create a softlink from your package library, say, /usr/local/lib/R/site-library/myPackage/scripts/someScript.R to a directory in the $PATH, say /usr/local/bin.
The key aspect is that the softlink persists even as you update the package. So now you are golden. All you now need is your crontab entry referencing someScript.R. We use a mix of Rscript and littler scripts.