Is it possible to limit the hbase shell to allow just the "read only" commands?
The goal is to avoid inadvertent changes to the data. I am imagining something similar to the readonly connections of many SQL databases.
Related
I tend to run simulations on a cluster that produces files larger than 100MB and I can't sync my computer with the cluster. So I considered setting up rsync between the two by following this link.
However, I believe this is just a cron job to sync the backup server with the main server and doesn't work in both directions. What will be the stepwise instructions to set up a bidirectional rsync ?
Both the systems run linux
Rsync isn't really the right tool for this job. You can sort of get it to work, using cron jobs and extremely carefully chosen parameters, but there's significant danger of data loss, especially if you want file deletion to propagate.
I'd recommend a tool like Syncthing for bidirectional sync. You want something that maintains an independent database of what's changed and what hasn't, and real-time updates are nice to have too.
I am trying to connect to Progress DB 4GL to extract data using ODBC (Qlikview).
Is is possible to extract data using 4GL via ODBC?
Can you please guide me on this?
Thanks and Regards,
Silambarasan
You can use the included ODBC driver but that uses SQL syntax, not ABL (4GL) queries.
Progress Datadirect has a product that offers an ODBC frontend for OpenEdge Appserver https://documentation.progress.com/output/DataDirect/openaccessbooks/oaoeguide.pdf
http://acorn.ro/products/abl-jdbc/ offers a jdbc interface to ABL code.
ODBC clients talk SQL, not 4GL.
To use the 4GL to extract data you use either a "self service" (shared memory) connection if you are on the same host as the server, or you can use a "client networking" license if you are on a network connected computer.
You could, of course, go through all sorts of convoluted stuff to use the 4GL without an actual db connection and a series of intermediate layers to somehow get an ODBC data connection. (You could, for instance, use sockets to communicate with a C# app that uses ODBC to connect to Progress...) But the specifics would depend on what you have available, what your coding skills are and why you are trying to do this.
If you were to manage to cobble together something like that you would still have the problem that the ODBC connection speaks SQL. So if you are expecting features like 4GL index selection or triggers or "variable width data everywhere" to work you are going to be out of luck. The SQL interface is ignorant of those things and won't be able to provide them.
I'm pretty sure that Clikview doesn't talk 4GL at all -- so you would also have to convert from 4GL back to SQL again. The whole thing makes no real sense.
(Regarding "why" I can only imagine that you are attempting to bypass a licensing requirement. If that is the case then you are not only making this much, much more difficult than it needs to be but you are also not going to solve the licensing problem. The license doesn't care what technologies and products are, or are not, used to make connections.)
As far as I know, you can use the Progress SQL92 interface via ODBC to access 4GL tables (CRUD for records), but you cannot create or amend 4GL tables using SQL92.
Any tables you create using SQL92 CREATE TABLE command, cannot be seen by 4GL/ABL programs.
The Progress website says that the Progress ODBC driver is included in the Progress/Openedge client package.
I was playing with percona xtrabackup innobackupex for incremental backups. It is a cool tool and very efficient and effective for incremental backups. However, i could not help but wonder why doing incremental backups would be any better than just doing a regular mysql master-slave replication, and whenever needed to retrieve point-in-time data, just use the binary log?
What advantages would doing incremental backups have over doing master-slave replication? When should you choose to use over the other?
One disadvantage to using master-slave replication as a backup is that accidentally running data damaging commands like
DROP TABLE users;
would replicate to the slave.
They are solutions to two different problems; master-slave is redundancy and backup is resilience.
The MySQL JDBC driver has the ability to connect to many servers. If you look at the driver options (https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-url-format.html) you will notice that the host option is not only host, but hosts. If you specify the URL to both the master and the slave and something happens to the master the driver will automatically connect to the slave instead.
Backup, on the other hand, is, as was mentioned earlier, a way to recover from either a catastrophic crash (having your backups stored off-site is a must) or recover from a catastrophic mistake -- neither of which is served by a master-slave setup. (Well, technically you could have the slave at a different site but that still does not cover the mistake scenario)
We have a IBM Host System Z sitting in our cellar. Now the issue is that i have no clue about Mainframes!!! (It's not USS btw.)
The Problem: How can i transfer a file from the host system to a windows machine.
Usually on UNIX systems i would just install and ssh daemon and connect to it via. a program called winscp. After that transfer the file in binary so that it does not convert something (Ultraedit and other Editors can handle this).
With the host system it seems to be a bit difficult as the original format from IBM is EBCDIC and i have no idea if there is a state of the art SFTP server program for the host. Could anybody be so kind and enlighten me? From my current expirience with IT there must be a state of the art sftp connection to that system? I appreciate any help/hints/solutions.
Thank you,
O.S
If the mainframe "sitting in [your] cellar" is running z/OS then it has Unix System Services installed. You can't have z/OS without it.
There is an SFTP package available (for free) for z/OS.
You can test to see about Unix System Services by firing up a 3270 emulator going to ISPF option 3.17, putting a forward slash (/) in the Pathname field and pressing the mainframe Enter key. Another way would be to key OMVS at a TSO READY prompt, which will start up a 3270-based Unix shell.
It is possible that USS is simply not available to you; if you're running any supported release of z/OS then USS is present. There could be concerns about supporting something outside a particular group,
Or, depending on what OS you have running on your System z, it's possible you don't have z/OS. You could have z/VM, you could have zLinux, you could have TPF. However, if you're running zLinux, you have linux, which has sftp installed, and which uses ASCII, not EBCDIC.
As cschneid says, however, if you have z/OS, you have USS. TCP/IP, among other things, won't run without it. Also note that z/OS TCP/IP has an FTP server, so you can connect that way if the FTP server is set up. If security is an issue, FTPS is supported, although it's painful to set up. With the native FTP server, you can convert from EBCDIC to ASCII when you're doing the transfer. There's also an NFS server available. And SMB as well, I believe.
And there's an FTP client available as well, so you could FTP from z/OS to your system, if you wanted to.
Maybe a better thing to do would explain what you're trying to do with the data, and what the data is, in general. You can edit files directly on the mainframe, using either TSO, ISPF, or OMVS editors. There are a lot of data types that the mainframe supports that you're not going to be able to handle on a non-z system unless you go through an export process. I'm not really clear on whether you want to convert the file to ASCII when you transfer it or not.
While the others are correct that all recent releases of z/OS have USS built-in, there's quite a bit of setup work that needs to be done in order for individual users to have access to USS capabilities like SFTP. Out of the box, you get USS "minimal mode" that just has enough of USS to support the TCP/IP stack and so forth. USS "full function mode" requires setup:
HFS filesystems need to be allocated
Your security package needs to be manage UIDs/GIDs for your users
etc etc etc
Still, with these details and with nothing more than the software you're entitled to as part of your z/OS license, you can certainly run SFTP and all the other UNIX style network services you're used to.
A good place to start is the UNIX Services Planning guide: http://publibz.boulder.ibm.com/epubs/pdf/bpxzb2c0.pdf
I'm having a locking problem where an SQLITE3 databse is permanently locked when created on an NFS file system. I have read that an option called nobrl can help this issue when the file system in question is CIFS. (its an option to the mount command).
From: http://linux.die.net/man/8/mount.cifs
nobrl
Do not send byte range lock requests to the server. This is
necessary for certain applications that break with cifs style
mandatory byte range locks (and most cifs servers do not yet support
requesting advisory byte range locks).
Is there any way to stop byte-range-lock requests in NFS if they occur, or am I running in the wrong direction by even thinking about this? I'm happy to change the mount command as was done for the CIFS solution.
I recommend to open you sqlite db by software with nolock parameter enabled, golang exg.:
sql.Open("sqlite3", "file:/media/R/Databases//your.db?nolock=1")
while /media/R is a mounted windows nfs-network-drive. Be carefull because you have to lock your db interactions by software otherwise you could corrupt your db, when accessing it simultaneously.
You can read more about sqlite parameters here:
https://www.sqlite.org/c3ref/open.html