I was reading a MS Excel help article about pivotcache and wonder what they mean by OLE DB and ODBC sources
...You should use the CommandText
property instead of the SQL property,
which now exists primarily for
compatibility with earlier versions of
Microsoft Excel. If you use both
properties, the CommandText property’s
value takes precedence.
For OLE DB sources, the CommandType
property describes the value of the
CommandText property.
For ODBC sources, the CommandText
property functions exactly like the
SQL property, and setting the property
causes the data to be refreshed...
I really appreciate your short answers.
According to ADO: ActiveX Data Objects, a book by Jason T. Roff, published by O'Reilly Media in 2001 (excellent diagram here), he says precisely what MOZILLA said.
(directly from page 7 of that book)
ODBC provides access only to relational databases
OLE DB provides the following features
Access to data regardless of its format or location
Full access to ODBC data sources and ODBC drivers
So it would seem that OLE DB interacts with SQL-based datasources THRU the ODBC driver layer.
I'm not 100% sure this image is correct. The two connections I'm not certain about are ADO.NET thru ADO C-api, and OLE DB thru ODBC to SQL-based data source (because in this diagram the author doesn't put OLE DB's access thru ODBC, which I believe is a mistake).
ODBC:- Only for relational databases (Sql Server, Oracle etc)
OLE DB:- For both relational and non-relational databases. (Oracle, Sql-Server, Excel, raw files, etc)
Here's my understanding (non-authoritative):
ODBC is a technology-agnostic open standard supported by most software vendors.
OLEDB is a technology-specific Microsoft's API from the COM-era (COM was a component and interoperability technology before .NET)
At some point various datasouce vendors (e.g. Oracle etc.), willing to be compatible with Microsoft data consumers, developed OLEDB providers for their products, but for the most part OLEDB remains a Microsoft-only standard. Now, most Microsoft data sources allow both ODBC and OLEDB access, mainly for compatibility with legacy ODBC data consumers. Also, there exists OLEDB provider (wrapper) for ODBC which allows one to use OLEDB to access ODBC data sources if one so wishes.
In terms of the features OLEDB is substantially richer than ODBC but suffers from one-ring-to-rule-them-all syndrome (overly generic, overcomplicated, non-opinionated).
In non-Microsoft world ODBC-based data providers and clients are widely used and not going anywhere.
Inside Microsoft bubble OLEDB is being phased out in favor of native .NET APIs build on top of whatever the native transport layer for that data source is (e.g. TDS for MS SQL Server).
ODBC and OLE DB are two competing data access technologies. Specifically regarding SQL Server, Microsoft has promoted both of them as their Preferred Future Direction - though at different times.
ODBC
ODBC is an industry-wide standard interface for accessing table-like data. It was primarily developed for databases and presents data in collections of records, each of which is grouped into a collection of fields. Each field has its own data type suitable to the type of data it contains.
Each database vendor (Microsoft, Oracle, Postgres, …) supplies an ODBC driver for their database.
There are also ODBC drivers for objects which, though they are not database tables, are sufficiently similar that accessing data in the same way is useful. Examples are spreadsheets, CSV files and columnar reports.
OLE DB
OLE DB is a Microsoft technology for access to data. Unlike ODBC it encompasses both table-like and non-table-like data such as email messages, web pages, Word documents and file directories. However, it is procedure-oriented rather than object-oriented and is regarded as a rather difficult interface with which to develop access to data sources. To overcome this, ADO was designed to be an object-oriented layer on top of OLE DB and to provide a simpler and higher-level – though still very powerful – way of working with it. ADO’s great advantage it that you can use it to manipulate properties which are specific to a given type of data source, just as easily as you can use it to access those properties which apply to all data source types. You are not restricted to some unsatisfactory lowest common denominator.
While all databases have ODBC drivers, they don’t all have OLE DB drivers. There is however an interface available between OLE and ODBC which can be used if you want to access them in OLE DB-like fashion. This interface is called MSDASQL (Microsoft OLE DB provider for ODBC).
SQL Server Data Access Technologies
Since SQL Server is (1) made by Microsoft, and (2) the Microsoft database platform, both ODBC and OLE DB are a natural fit for it.
ODBC
Since all other database platforms had ODBC interfaces, Microsoft obviously had to provide one for SQL Server. In addition to this, DAO, the original default technology in Microsoft Access, uses ODBC as the standard way of talking to all external data sources. This made an ODBC interface a sine qua non.
The version 6 ODBC driver for SQL Server, released with SQL Server 2000, is still around. Updated versions have been released to handle the new data types, connection technologies, encryption, HA/DR etc. that have appeared with subsequent releases. As of 09/07/2018 the most recent release is v13.1 “ODBC Driver for SQL Server”, released on 23/03/2018.
OLE DB
This is Microsoft’s own technology, which they were promoting strongly from about 2002 – 2005, along with its accompanying ADO layer. They were evidently hoping that it would become the data access technology of choice. (They even made ADO the default method for accessing data in Access 2002/2003.) However, it eventually became apparent that this was not going to happen for a number of reasons, such as:
The world was not going to convert to Microsoft technologies and
away from ODBC;
DAO/ODBC was faster than ADO/OLE DB and was also thoroughly integrated into MS Access, so wasn’t going to die a natural death;
New technologies that were being developed by Microsoft, specifically ADO.NET,
could also talk directly to ODBC. ADO.NET could talk directly to OLE
DB as well (thus leaving ADO in a backwater), but it was not (unlike
ADO) solely dependent on it.
For these reasons and others, Microsoft actually deprecated OLE DB as a data access technology for SQL Server releases after v11 (SQL Server 2012). For a couple of years before this point, they had been producing and updating the SQL Server Native Client, which supported both ODBC and OLE DB technologies. In late 2012 however, they announced that they would be aligning with ODBC for native relational data access in SQL Server, and encouraged everybody else to do the same. They further stated that SQL Server releases after v11/SQL Server 2012 would actively not support OLE DB!
This announcement provoked a storm of protest. People were at a loss to understand why MS was suddenly deprecating a technology that they had spent years getting them to commit to. In addition, SSAS/SSRS and SSIS, which were MS-written applications intimately linked to SQL Server, were wholly or partly dependent on OLE DB. Yet another complaint was that OLE DB had certain desirable features which it seemed impossible to port back to ODBC – after all, OLE DB had many good points.
In October 2017, Microsoft relented and officially un-deprecated OLE DB. They announced the imminent arrival of a new driver (MSOLEDBSQL) which would have the existing feature set of the Native Client 11 and would also introduce multi-subnet failover and TLS 1.2 support. The driver was released in March 2018.
• August, 2011: Microsoft deprecates OLE DB (Microsoft is Aligning with ODBC for Native Relational Data Access)
• October, 2017: Microsoft undeprecates OLE DB (Announcing the new release of OLE DB Driver for SQL Server)
On a very basic level those are just different APIs for the different data sources (i.e. databases). OLE DB is newer and arguably better.
You can read more on both in Wikipedia:
OLE DB
ODBC
I.e. you could connect to the same database using an ODBC driver or OLE DB driver. The difference in the database behaviour in those cases is what your book refers to.
Both are data providers (API that your code will use to talk to a data source). Oledb which was introduced in 1998 was meant to be a replacement for ODBC (introduced in 1992)
I'm not sure of all the details, but my understanding is that OLE DB and ODBC are two APIs that are available for connecting to various types of databases without having to deal with all the implementation specific details of each. According to the Wikipedia article on OLE DB, OLE DB is Microsoft's successor to ODBC, and provides some features that you might not be able to do with ODBC such as accessing spreadsheets as database sources.
At Microsoft website, it shows that native OLEDB provider is applied to SQL server directly and another OLEDB provider called OLEDB Provider for ODBC to access other Database, such as Sysbase, DB2 etc. There are different kinds of component under OLEDB Provider. See Distributed Queries on MSDN for more.
ODBC works only for relational databases, it can't works with non-relational databases such as Ms Excel files. Where Olebd can do everything.
To know why M$ invents OLEDB, you can't compare OLEDB with ODBC. Instead, you should compare OLEDB with DAO,RDO, or ADO. The latter largely relies on SQL. However, OLEDB relies on COM. But ODBC is already there many years, so there's a OLEDB-ODBC bridges to remedy this. I think there's a big picture when M$ invents OLEDB.
Related
I am looking for a solution to stream data from Oracle 11g to Kafka. I was hoping to use GoldenGate, but that only seems to be available for Oracle 12c. Is the Confluent platform the best way to go?
Thanks!
First, the general answer would be: The best way to connect Oracle (databases) to Kafka is indeed to use Confluent Platform with Kafka's Connect API in combination with a ready-to-use connector for GoldenGate. See the GoldenGate/Oracle entry in section "Certified Connectors" at https://www.confluent.io/product/connectors/. The listed Kafka connector for GoldenGate is maintained by Oracle.
Is the Confluent platform the best way to go?
Hence, in general, the answer to the above question is: "Yes, it is."
However, as you pointed out for your specific question about Oracle versions, Oracle unfortunately has the following information in the README of their GoldenGate connector:
Supported Versions
The Oracle GoldenGate Kafka Connect Handler/Formatter is coded and
tested with the following product versions.
Oracle GoldenGate for Big Data 12.2.0.1.1
Confluent IO Kafka/Kafka Connect 0.9.0.1-cp1
Porting may be required for Oracle GoldenGate Kafka Connect
Handler/Formatter to work with other versions of Oracle GoldenGate for
Big Data and/or Confluent IO Kafka/Kafka Connect
This means that the connector does not work with Oracle 11g, at least as far as I can tell.
Sorry if that doesn't answer your specific question. At least I wanted to give you some feedback on the general approach. If I do come across a more specific answer, I'll update this text.
Update Mar 15, 2017: The best option you have at the moment is to use the Confluent's JDBC connector. That connector can't give you quite the same feature set as Oracle's native GoldenGate connector though.
Oracle GoldenGate and Confluent Platform are not comparable.
Confluent Platform provides the complete streaming platform and is a collection of multiple software which can be used for streaming your data, where as GoldenGate is replication and data-integration software.
Also GoldenGate is highly reliable for db replication since it maintains transactional integrity, same cannot be said for Kafka Mirror Maker or Confluent's Replicator at this time.
If you want just pure transactions - please also consider using OpenLogReplicator. It supports Oracle database from version 11.2.0.1.
It can produce transactions to Kafka in 2 formats:
Classic format - when every transaction is one Kafka message (multiple DMLS per Kafka message)
Debezium style format - transactions are divided - every DML is one Kafka message
There is already a working version. You can try it.
Right now I am using ojdbc6 to connect to Oracle 11g. It is good enough but not perfect especially when using pooling mode to check if there are new updates on the original tables.
I tried also to read all tables using certain pattern but this did not work well.
The best mode to connect an Oracle DB to Kafka (especially when the tables are very wide, columns wise, is to use queries for the connectors. This way, you ensure that you pick the right fields and do some casting for numbers if you are using avro.
I am trying to connect to Progress DB 4GL to extract data using ODBC (Qlikview).
Is is possible to extract data using 4GL via ODBC?
Can you please guide me on this?
Thanks and Regards,
Silambarasan
You can use the included ODBC driver but that uses SQL syntax, not ABL (4GL) queries.
Progress Datadirect has a product that offers an ODBC frontend for OpenEdge Appserver https://documentation.progress.com/output/DataDirect/openaccessbooks/oaoeguide.pdf
http://acorn.ro/products/abl-jdbc/ offers a jdbc interface to ABL code.
ODBC clients talk SQL, not 4GL.
To use the 4GL to extract data you use either a "self service" (shared memory) connection if you are on the same host as the server, or you can use a "client networking" license if you are on a network connected computer.
You could, of course, go through all sorts of convoluted stuff to use the 4GL without an actual db connection and a series of intermediate layers to somehow get an ODBC data connection. (You could, for instance, use sockets to communicate with a C# app that uses ODBC to connect to Progress...) But the specifics would depend on what you have available, what your coding skills are and why you are trying to do this.
If you were to manage to cobble together something like that you would still have the problem that the ODBC connection speaks SQL. So if you are expecting features like 4GL index selection or triggers or "variable width data everywhere" to work you are going to be out of luck. The SQL interface is ignorant of those things and won't be able to provide them.
I'm pretty sure that Clikview doesn't talk 4GL at all -- so you would also have to convert from 4GL back to SQL again. The whole thing makes no real sense.
(Regarding "why" I can only imagine that you are attempting to bypass a licensing requirement. If that is the case then you are not only making this much, much more difficult than it needs to be but you are also not going to solve the licensing problem. The license doesn't care what technologies and products are, or are not, used to make connections.)
As far as I know, you can use the Progress SQL92 interface via ODBC to access 4GL tables (CRUD for records), but you cannot create or amend 4GL tables using SQL92.
Any tables you create using SQL92 CREATE TABLE command, cannot be seen by 4GL/ABL programs.
The Progress website says that the Progress ODBC driver is included in the Progress/Openedge client package.
I need to connect Tableau to HBase or Phoenix and Tableau does not support JDBC. Bummer!
I've read about the proprietary Simba driver but haven't seen any reports of people using it. I don't feel like forking over money when it's not ideal, and my employer feels the same way.
Is there another way to connect Tableau to HBase or Phoenix? How are other people doing it? I don't like the idea of using Hive to connect to HBase because one of the main reasons to go away from Hive is its atrocious performance, so I hope that's not my 'best' alternative.
On the other hand, if people have used Simba and it works well, I'm curious to hear about that.
I am the developer on Simba's Phoenix driver. Hortonworks, Cloudera, Databricks, Microsoft, Amazon, Google, etc all choose Simba's drivers for a variety of products.
ie. http://hortonworks.com/partner/simba/,
http://www.simba.com/news/databricks-offers-simba-technologies-developed-odbc-3-8-connectivity-sql-capability-apache-spark/
Also, you need to choose either Phoenix on HBase or HBase standalone for all of your applications. The two types of drivers encode data in different binary representations. String and unsigned integers will be interpreted correctly, but unsigned integers and more complex data-types will be decoded differently.
ie. Phoenix doesn't display negative integer values correctly
So if you use Phoenix JDBC for your / applications, you cannot use an HBase ODBC driver with Tableau (unless you have nothing but strings and unsigned integers in your datasource). From your other postings, you do use Phoenix JDBC. So HBase ODBC is not an option for you.
The CData ODBC driver will allow you to connect Tableau to HBase (full disclosure: I work for CData Software). You can download a free Beta here. We have an article that outlines the configuration and connection, but I've copied the relevant information steps:
Create/configure a DSN from the ODBC Driver by setting the server address and port (we use the REST API, so the default port is 8080)
You should click the "Test Connection" button in the DSN Configuration wizard to ensure that you can establish a proper connection to your HBase database.
Click through the "Connect to Data" options to find "Other Database (ODBC)" and select the DSN you configured
Select CData as the database
Enter a Table name (or leave the Table field blank and click search (the magnifying glass) to see a list of Tables).
Once you have access to the tables, you can work with them exactly as you would any other table in Tableau (drag the table to the join area, manipulate Measures and Dimensions to view your data, etc.). If you have any questions, I or our Support Team will be happy to help.
I want to be able to query an Exchange address book via ODBC. It seems like there ought to be a driver available for this puprose but I can't find one.
MS Access can link the Exchange table, and I can then query the Access database via ODBC, but it is pitifully slow.
For the record, I'm not programming so I don't need ADO connection strings or whathaveyou. The database software I'm using (Drawbase, a space database for facilities management) needs an ODBC system data source so I need an ODBC appropriate driver so I can create one.
There is one here:
http://www.connect-gate.com/index.php/en/why-connect-gate/sql-for-communication
It's quite fast and cover the most important SQL statements.
I've done a fair amount of ODBC work and also run Exchange and I've never heard of one. :(
CData seems to provide the driver now: https://www.cdata.com/drivers/exchange/odbc/ - not sure if this is the same as the Connect Gate driver above or not.
hoping someone has experience with both sql server and postgresql.
Between the two db's, which one is easier to scale?
Is creating a read only db that mirrors the main db easier/harder than sql server?
Seeing as sql server can get $$, I really want to look into postgresql.
Also, are the db access libraries written well for an asp.net application?
(please no comments on: do you need the scale, worry about scaling later, and don't optimize until you have scaling issues...I just want to learn from a theoretical standpoint thanks!)
Currently, setting up a read-only replica is probably easier with SQL Server. There's a lot of work going on to get hot standby and streaming replication part of the next release, though.
Regarding scaling, people are using PostgreSQL with massive databases. Skype uses PostgreSQL, and Yahoo has something based on PostgreSQL with several petabyte in it.
I've used Postgresql with C# and ASP.Net 2.0 and used the db provider from devart:
http://www.devart.com/dotconnect/postgresql/
The visual designer has a few teething difficulties but the connectivity was fine.
I have only used SQL Server and not much PostgreSQL, so I can only answer for SQL Server.
When scaling out SQL Server you have a couple of options. You can use peer to peer replication between databases, or you can have secondary read-only DB(s) as you mention. The last option is relatively straight-forward to set up using database mirroring or log shipping. Database mirroring also gives you the benefit of automatic switchover on primary DB failure. For an overview, look here:
http://www.microsoft.com/sql/howtobuy/passive-server-failover-support.mspx
http://technet.microsoft.com/en-us/library/cc917680.aspx
http://blogs.technet.com/josebda/archive/2009/04/02/sql-server-2008-database-mirroring.aspx
As for licensing, you only need a license for the standby server if it is actively used for serving queries - you do not need one for a pure standby server.
If you are serious, you can set up a failover cluster with a SAN for storage, but that is not really a load balancing setup in itself.
Here are some links on the general scale up/out topic:
http://www.microsoft.com/sqlserver/2008/en/us/wp-sql-2008-performance-scale.aspx
http://msdn.microsoft.com/en-us/library/aa479364.aspx
ASP.NET Libraries are obviously very well written for SQL Server, but I would believe there exists good alternatives for PostgreSQL as well.