I run BlacBox Component Builder under Wine/Ubuntu. I need to set up ODBC in order to access databases (PostgresQL in my case) in BBCB programs (Component Pascal). They have a unique and wonderful SQL access library that allows to accept SELECT results directly into a record variable (analogous to structures in the C world or class instances in C++). Such record is called an interactor. No preprocessor is used, and query parameters are taken directly out of program variables (Like so: "SELECT * FROM users WHERE name = :u.name AND age < :maxage", where u is a program variable of type RECORD and .name is it's field, maxage is also a program variable). Anyway, very convenient from a programmer's perspective.
However, BBCB does not have any native DB drivers for Linux, so I have to use the Windows BBCB version in Wine. Some years ago, I was able to set up unixodbc in Ubuntu, and unixodbc's DSNs became immediately available in Wine. Not this time around.
So, the question is, How do I set up ODBC in Ubuntu so that it's DSNs would be visible in Wine?
I found the trick!
BBCB is a 32-bit app, so obviousely it makes calls to 32-bit WINE/winapi functions, and they seemingly can't talk to the unixodbc/64bit.
The trick was to install :i386 versions of unixodbc and the DB odbc driver, then create a 32-bit wine prefix, and then run my app from there. In greater detail:
If not already done,
dpkg --add-architecture i386
sudo apt install unixodbc:i386
install database driver for odbc for your database server; in my case,
sudo apt install postgresql-odbc:i386
Assuming ~/.wine32 does not exist:
WINEPREFIX=~/.wine32 WINEARCH=win32 wine wineboot
(some manuals insist you have to have a wine 64-bit prefix before this)
Now copy BlackBox from it's previous location, or install anew, into the ~/.wine32 prefix, and from now on run BB as follows:
WINEPREFIX=~/.wine32 WINEARCH=win32 <program.exe-file-location>
Now BB's SqlOdbc module can perfectly see the DSNs I set up in unixodbc.
Related
I need my application to run from a USB stick and perform the installation from there.
The application is eventually installed on a Linux/Debian.
For the application installation I need a DB to be installed on that USB. I also need the DB data (tables, etc.) to be kept on that USB stick.
I read that SQLite is a good candidate to be used for such a purpose. However, I could no find the steps needed for installing it on the USB stick.
I did download the sqlite-snapshot-202002271621.tar.gz from the sqlite.org site, placed it in one of my Debian directories and used the 3 commands to install it (./configure,make,make install).
That installed SQLite on my hard disk.
What should I do in order to achieve the same on the USB stick?
Mount the USB to the Debian box, place the tar.gz file there, and run the commands from there?
Will that install SQLite on the USB?
Thanks
So the answer is indeed:
Mount the USB to the Debian box
Place the sqlite-snapshot-202002271621.tar.gz file there
Run the commands from there
./configure
make
make install
Notice just that I had to tar the sqlite-snapshot-202002271621.tar.gz file using:
tar -zvxf sqlite-snapshot-202002271621.tar.gz -C /media/usbstick/ --no-same-owner
In order to avoid the error:
"Cannot change ownership to uid 1000, gid 1000: Operation not permitted"
I got my connection from R to a MS SQL Server database working on a Mac. I was able to get the right driver for the connection string by issuing the odbcListDrivers() command. However, I don't know if it was there or not before I started installing things like freetds and brew install --no-sandbox msodbcsql17 mssql-tools.
What I'd like to know is: with the odbcListDrivers(), how do I know what the file is it pulls from (a .ini one, I'm thinking)? I struggled because the configuration files I configured for freetds and another odbc.ini file don't affect what R is reading I believe, and made troubleshooting difficult.
In case I need to configure more drivers or DSN (?) entries.
How can I connect and run queries against AWS RedShift using .NET Core? Code sample please. I have gone through the AWS docs and .Net Core docs but no luck.
This answer is one for a particular point in time and won't age well...
The EntityFramework Core project is the one I'd keep the closest eye on. The lack of ODBC is well known, especially for those who want to connect to Oracle. For the time being, you may need to fork an Oracle client for .NET core and make modifications as necessary.
I found these projects after a quick Google search that may be able to help you for now...
- https://github.com/LinqDan/oracleclientcore
- https://github.com/LinqDan/Mono.Data.OdbcCore
Longer term, you'll want to keep an eye on these two GitHub issues which are tracking it for EntityFramework Core and the .NETStandard APIs..
https://github.com/dotnet/corefx/issues/13035
https://github.com/aspnet/EntityFramework/issues/7432
Update 6/23/2017:
This is now possible through the Npgsql.EntityFrameworkCore.PostgreSQL NuGet package and associated Entity Framework Core packages. Apparently the PostgreSQL team was tired of waiting around for ODBC support (which still isn't available in the latest netstandard2.0 yet) and wrote their own driver using netstandard - back in the November timeframe. The getting started page on the npgsql website covers it's usage in the old JSON project format - but the dependencies listed are still valid.
Here is how you would use the package...
using (var conn = new NpgsqlConnection("Host=myserver;Username=mylogin;Password=******;Database=music"))
{
conn.Open();
using (var cmd = new NpgsqlCommand())
{
cmd.Connection = conn;
cmd.CommandText = "SELECT name FROM artists";
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
Console.WriteLine(reader.GetString(0));
}
}
}
}
One thing to keep in mind when using this driver - it's written for PostgreSQL, not Redshift. While Redshift is based on PostgreSQL, it's underlying engine is much more like Cassandra than anything else. As a result, Amazon had to make some choices in the development to drop certain things that PostgreSQL does support - such as SQL variables. Because of this, you will have a fairly limiting experience for certain things that you might be used to in other Entity Framework implementations. As long as you stay with using the direct access *Connection, *Command, and DataReader classes and write your own SQL, you should be fine.
At the time of writing this answer there are ODBC drivers readily available for Redshift.
For Windows, you can find the "Amazon Redshift" driver installation file here:
https://docs.aws.amazon.com/redshift/latest/mgmt/install-odbc-driver-windows.html
This will need to be installed on the machine (or machines) that run the application required to connect to Redshift. For automated deployments, it would be a good idea to have a powershell script (or MSBuild step) that checks if it is installed on the production machine and install it if it doesn't exist.
For Linux, the driver can be found here:
https://docs.aws.amazon.com/redshift/latest/mgmt/install-odbc-driver-linux.html
Since most of the world of .NET Core is moving towards using Linux docker containers I will elaborate further on how to set this up in a docker environment using a Dockerfile.
First you will need to create a shell script (e.g. install-driver.sh) that does an apt-get for the relevant odbc driver you want from the link above.
Something like this:
#!/bin/bash
# Install the Redshift ODBC driver manager
apt-get update \
&& apt-get install -y --no-install-recommends unixodbc
if ! curl -s https://s3.amazonaws.com/redshift-downloads/drivers/odbc/[latestversion].deb -o driver.deb; then
echo 'Failed to download Redshift ODBC Driver!' 1>&2
exit 1
fi
# Install the Redshift ODBC driver
apt install ./driver.deb
There are also three files required for configuring the Amazon Redshift ODBC driver on Linux: amazon.redshiftodbc.ini, odbc.ini, and odbcinst.ini. This is explained here:
https://docs.aws.amazon.com/redshift/latest/mgmt/odbc-driver-configure-linux-mac.html
As mentioned in that link, you will have to add the following lines to your .sh file:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export ODBCINI=/etc/odbc.ini
export AMAZONREDSHIFTODBCINI=/etc/amazon.redshiftodbc.ini
export ODBCSYSINI=/usr/local/odbc
Once you have created the .sh and .ini files put them into a common directory (e.g. /odbc/) and include them as part of your Dockerfile commands. Something like this:
FROM microsoft/dotnet:2.1-sdk AS base
WORKDIR /app # or wherever your working directory resides
COPY ./odbc/odbc.ini /etc/odbc.ini
COPY ./odbc/odbcinst.ini /etc/odbcinst.ini
COPY ./odbc/amazon.redshiftodbc.ini /etc/amazon.redshiftodbc.ini
COPY ./odbc/install-driver.sh /tmp/install-driver.sh
RUN chmod +x /tmp/install-driver.sh
RUN /tmp/install-driver.sh
# dotnet restore, build, publish and ENTRYPOINT commands here...
That should be enough for you to run the docker build and docker run commands with the Redshift driver installed within your docker container.
It means that your application code can use OdbcConnection just like any other database driver:
using System;
using System.Data;
using System.Data.Odbc;
// field within class... value should be set from config
private string redshiftConnectionString = "Driver={Amazon Redshift (x64)}; Server=redshiftclusterhostname.region.redshift.amazonaws.com; Database=database; UID=user; PWD=password; Port=5439"
public DataSet ExecuteQuery(string query)
{
var dataSet = new DataSet();
using (var connection = new OdbcConnection(this.redshiftConnectionString))
{
var adapter = new OdbcDataAdapter(query, connection);
adapter.Fill(dataSet);
}
return dataSet;
}
Take note that the Driver= value should be the name of the driver you chose to download and install.
Simply connect to Amazon Redshift via a JDBC or ODBC Driver and access it like a normal SQL database.
You'll use the AWS API to start/stop the cluster, but all queries and requests go via the SQL connection.
I'm new to R and I decided to put R on a machine I have and see if I can remotely run code that is on my desktop computer.
While searching for "how to do" that, I came across the names "Rserve" and "RStudio". As far as I could tell, RServe is a package (actually, it seems to be the package) which I can use to configure the server, while RStudio is an IDE.
My question is: does RStudio use RServe "under the hood"? And, if it doesn't, then how does RStudio compare to RServe? (I.e., which one is better and why?)
[I figured out that this question could possibly be a duplicate, but I couldn't find any similar question]
Rserve is a client server implemenation written in pure c that starts a server and spawns multiple processes each with it's own R workspace. This is not threads but processes due to R's limitation on multithreading. It uses a QAP packing protocol as it's primary form of transport between the client and the server. You execute commands via the client (PHP, Java, C++) to the server and it returns you REXP objects that are essentially mappings to R's underlying SEXP data objects. Rserve also offers a websockets version that does will can transmit data through websockets but the api is not well documented. It also supports basic authentication through a configuration file.
Rstudio is a C++ and gwt application that provides a web based front end to R. AFAIK it uses json as it's primary transport and supports authentication through pam. Each user has a workspace configured in their home directory. It runs a server very similar but not the same as Rserve to communicate with R using RCPP. It also has it's own plotting driver used to wrap the plot device so that it can pickup the plots to be served to the ui. It has much more functionality such as stepping through your code from the ui and viewing workspace variables.
Functionally they are similar in that they provide a client/server connection to R but IMHO the comparison stops there.
I believe they are separate projects (though I could be wrong). I've never heard of RServe and there does not appear to be any mention of it in the documentation for RStudio. I have used and would recommend RStudio Server. It is relatively easy to set up and super easy to use once it is set up. This is a helpful guide to setting up a server on Amazon EC2:
#Create a user, home directory and set password
sudo useradd rstudio
sudo mkdir /home/rstudio
sudo passwd rstudio
#Enter Password
sudo chmod -R 0777 /home/rstudio
#Update all files from the default state
sudo apt-get update
sudo apt-get upgrade
#Be Able to get R 3.0
sudo add-apt-repository 'deb http://cran.rstudio.com/bin/linux/ubuntu precise/'
#Update files to use CRAN mirror
#Don't worry about error message
sudo apt-get update
#Install latest version of R
#Install without verification
sudo apt-get install r-base
#Install a few background files
sudo apt-get install gdebi-core
sudo apt-get install libapparmor1
#Change to a writeable directory
#Download & Install RStudio Server
cd /tmp
wget http://download2.rstudio.org/rstudio-server-0.97.551-amd64.deb
sudo gdebi rstudio-server-0.97.551-amd64.deb
#Once you’ve installed the above commands, you can now access RStudio through your local browser. Navigate to the Public DNS of your image on port 8787, similar to:
#http://ec2-50-19-18-120.compute-1.amazonaws.com:8787
The earlier answer about 3 years old provide old information, such as here.
Updated correction
RStudio is a firm that provides the open source RStudio IDE for R. They also sell commercial services such as RStudio Server Pro that markets itself with load balancing and related things. Apparently, the successuful open source project has lead the way to markets.
You may also mean Microsoft R Server, which is now called Microsoft Machine Learning Server?
There is also RServer by RStudio.
Anyway how to install both can be found here.
Installing the RODBC package on Ubuntu is a bit of a kludge. First I learned to install the following:
$ sudo apt-get install r-cran-rodbc
That wasn't good enough as the package was still looking for header files. I solved this issue by:
$ sudo apt-get install unixodbc-dev
Good, RODBC installed properly on the Ubuntu machine. But when I try to run the following script:
## import excel file from Dropbox
require("RODBC")
channel <- odbcConnectExcel("~/Dropbox/DATA/SAMPLE/petro.xls")
petro <- sqlFetch (channel, "weekly")
odbcClose(channel)
str(petro)
head(petro)
I get an error thrown that function odbcConnectExcel not found. I checked the case of each letter, making sure it was not a simple typo. Nope. Then I ran this same script on a Windows R installation (file path different, of course) and the script works.
Any idea of why Ubuntu R installation cannot find the odbcConnectExcel function and how I can get this to work?
That functionality is available where Excel is available. In other words: not on Ubuntu.
For reference, from the R Data Import / Export manual (with my highlighting):
4.3.2 Package RODBC
Package RODBC on CRAN provides an
interface to database sources
supporting an ODBC interface. This is
very widely available, and allows the
same R code to access different
database systems. RODBC runs on
Unix/Linux, Windows and Mac OS X, and
almost all database systems provide
support for ODBC. We have tested
Microsoft SQL Server, Access, MySQL,
PostgreSQL, Oracle and IBM DB2 on
Windows and MySQL, Oracle, PostgreSQL
and SQLite on Linux.
ODBC is a client-server system, and we
have happily connected to a DBMS
running on a Unix server from a
Windows client, and vice versa.
On Windows ODBC support is normally
installed, and current versions are
available from
http://www.microsoft.com/data/odbc/ as
part of MDAC. On Unix/Linux you will
need an ODBC Driver Manager such as
unixODBC (http://www.unixODBC.org) or
iOBDC (http://www.iODBC.org: this is
pre-installed in Mac OS X) and an
installed driver for your database
system.
Windows provides drivers not just for
DBMSs but also for Excel (.xls)
spreadsheets, DBase (.dbf) files and
even text files. (The named
applications do not need to be
installed. Which file formats are
supported depends on the the versions
of the drivers.) There are versions
for Excel 2007 and Access 2007 (go to
http://download.microsoft.com, and
search for Office ODBC, which will
lead to AccessDatabaseEngine.exe), the
`2007 Office System Driver'.
I've found RODBC to be a real pain in the Ubuntu. Maybe it's because I don't know the right incantations, but I switched to RJDBC and have had much better luck with it. As discussed here.
As Dirk says, that wont solve your Excel problem. For writing Excel I've had very good luck with the WriteXLS package. In Ubuntu I found it quite easy to set up. I had Perl and many of the packages already installed and had to simply install Text::CSV_XS which I installed with the GUI package manager. The reason I like WriteXLS is the ability to write data frames to different sheets in the Excel file. And now that I look at your question I see that you want to READ Excel files not WRITE them. Hell. WriteXLS doesn't do that. Stick with gdata, like Dirk said in his comments:
gdata on CRAN and you are going to want the read.xls() function:
read.xls("//path//to/excelfile.xls", sheet = 1, verbose=FALSE, pattern, ...,
method=c("csv","tsv","tab"), perl="perl")
you may need to run installXLSXsupport which installs the needed Perl modules.
read.xls expect sheet numbers, not names. The method parameter is simply the intermediate file format. If your data has tabs then don't use tab as the intermediate format. And likewise for commas and csv.