I have an jdbc java class which will run jdbc queries and needs to be triggered from oozie workflow in hdfs.
currently my queries are part of the java code but i need to separate them into a properties file.
I can pass them as parameters, but is there any better way to available sql files directly to the java code in oozie action?
Dump the queries in text files
upload the files to HDFS
in "workflow.xml" reference each HDFS file in a <file> element so that Oozie downloads it in the Current Working Dir of the YARN container before starting your Java code
in "workflow.xml" reference each query file (in order) as <arg> of your Java "main()" -- with no path (remember, they will be present in the CWD)
in your Java code, iterate on args[] and load/execute each script
...
profit!
Related
I have a .ksh file from which I connect to Oracle DB at various environments using sqlplus as below
sqlplus -s $O_USER/$O_PASS#$O_DATABASE <<-EOF
Now, I need to read a properties(.txt) file from sqlplus for dynamically creating a url parameter and this file is located at client side. Is there any way to do this ? I'm ok with reading this via shell script and passing to sqlplus. I'm able to access some string variables in shell script from sqlplus, but is there a way to pass a hash map kind of object from shell script to sql plus ?
A few notes below :
I can't use UTL_FILE because the properties file should be located at client side only. Because I'm developing a monitoring tool for an application which is connected to the application in various environments and I don't want to or I don't have enough permission to put this properties file in each of those environments. So I want to store this properties file at a single place (client side)
I can't use TEXT_IO because I'm not using Oracle forms.
I don't want to put all these properties hard coded in the .ksh file (which will actually work) because there are more than 150 key-value pairs
I have a meteor project that runs a python script once a day to query an API and save the results to a JSON file in private. The meteor server watches that JSON file for changes. When the file changes, the server reads and parses the file and updates a collection accordingly.
The problem is that the assets in private are loaded at meteor startup and copied into a different asset folder as read-only, and thus the script can't make changes to the file.
I could maybe change the permissions on that asset destination folder but that seems hacky. I don't think assets in private are intended to be dynamic, anyways. Anybody know how I can accomplish this flow?
Meteor server kicks off python script once per day
Script queries API, saves results to JSON file on server
Meteor server reads JSON file and updates collection
The simplest solution may be for the Python script to write its JSON to a Mongo database. Then Meteor can automatically subscribe to any changes made to that collection.
Here is what I ended up doing:
Instead of having the meteor app kick off the python script daily, I just put the python script on the host's file system (outside of the meteor app bundle) and created a cron job on the host to run it daily. The JSON output file was saved to the host's file system.
Then, in my meteor app, I created a file watch on the output JSON that triggered a file read.
I was running into some issues with Meteor Up (mup), so see my other answered question here.
I have a server which runs multiple jar file at the same time as of now.
Currently we just make a bat file, call the java -jar xxxx.jar program, and the window is pop-ed up on the screen so we know which to terminate when we'd like to turn one of them off.
But as we progress we prefer those program to run at the background hence we'd prefer to use javaw -jar xxxx.jar instead.
However when we open up the task manager all it shows is many javaw.exe processes, without telling us which jar file its associated to.
Is there any parameter we can specify when we start javaw, so there's some indication on task manager's process list?
There is an official product named Process Explorer that can do what you want.
Is there C/C++ code or a Bash script that can sign Java .jar files without using a JVM?
Such code might be useful for embedded applications that serve a jar file with a small HTTP server, or other situations where a jar file might need to be edited and re-signed programmatically but there is insufficient storage space for a JVM.
Does anyone need this type of Javaless jarsigner?
I have an ASP.NET web application that includes code for enforcing its own database schema ; this code runs on application start.
I've recently started using LINQ to SQL, and I've added a pre-build event to run SqlMetal on my database so that I get objects representing my db tables.
What would be really cool is if I could enforce the database schema in the pre-build event, and then run SqlMetal. As it is, if the schema changes (e.g. I add a field to a table), I have to (a) build and run the website once so that application start fires and the schema is enforced, and then (b) build it again so that SqlMetal runs.
So: What are my options for running code that lives in my web application, from the command line?
Here's what we do.
We have a local 1-click build that is required to be run before check in (an integration build also runs in a separate environment every check in...).
The NANT script will:
Rebuild the database from scratch using Tarantino (Database change management)
Clean & Compile
Copy DLLs to a separate directory
Run Unit Tests against the DLLs
We have a separate script for SQL Metal, but your question is going to have me look at inserting the call between steps 1 and 2. This way your database changes and linq generated files are always in sync.
You could either write a small program that use CodeDOM to interpret a file in your repository or directly call the compiler executable inside your pre-build event.
Using CodeDOM avoid any problems with having to know where the compiler executable is but if your code can't be contained in one file without any dependency it's unusable and calling the compiler, then executing the result is a better option.