Teradata is exporting Time results in the Number format instead of the Time format. For example: 12:33 is exporting to 1233. However, while displaying the results in TeraData SQL assistant, the format is ok.
I've got the solution: If you extract the data using ODBC connection then it won't create trouble like you have to connect and select the server every time.
However if you connect using TeraData's default connection name then it will give you this issue.
Related
We need to use GenerateTableFetch to fetch huge data from teradata using nifi. But since Teradata is not present as a database type it generate Limit keyword. I am able to replace it with Sample keyword. But sample everytime give random value so how can i use Nifi with Teradata for huge table?
Maybe try the "MS SQL 2008" adapter, it generates a TOP statement as well as a ROW_NUMBER() OVER clause. If that doesn't work, we'd need a TeradataDatabaseAdapter implementation, please feel free to file an improvement Jira as needed.
We are using Teradata Fastexport connection in Informatica to export data from few tables by joining them in Source Qualifier query and write to a CSV file. We have around 180 columns to pull, recently we added 2 new columns to this flow and we found that data for few of the records looks junk. We figured out those records and tried to run the SQ query only for those records, to our surprise the columns which were junk earlier now throwing us expected data.
Is there any fast export limitation to export the columns ? or any properties should be increased at Informatica level.
We are clueless on this issue, please help.
I have a strange problem when using R ODBC to connect to an Access 2016 database.
The dates in R have a difference of -28 days by reference to the dates from the Access database.
For example, in Access: 1639-01-24; in R: 1638-12-27. The format of the date in R is "1638-12-27 LMT".
To solve this problem I had to get date variables as string with the Format function:
Select format([date],'yyyy-mm-dd') ...
Can someone explain why this is happening?
Despite the Date/Time field type in Access being defined as a "date or time value between the years 100 and 9999" (ref), the Access ODBC driver appears to have problems dealing with dates prior to January 1, 1753. I was able to confirm that using the Microsoft Access Driver (*.mdb, *.accdb) under both RODBC and .NET System.Data.Odbc.
1753-01-01 in Access is returned by Access ODBC as 1753-01-01 (correct), but
1752-12-31 in Access is returned by Access ODBC as 1752-12-30 (wrong)
and, as you have found, the discrepancy becomes larger as we go further back in time.
There is also a Microsoft Knowledge Base article here related to problems with Access ODBC when trying to insert dates prior to 1753.
It is probably not a coincidence that 1753-01-01 is the earliest possible date for the DATETIME column type in SQL Server. For more details on the significance of that date, see
What is the significance of 1/1/1753 in SQL Server?
There is only one Date/Time type in Access so we can't convert the value to something like DATETIME2 in SQL Server. Therefore the best workaround is probably to use Format() to convert the date value to text as mentioned in the question.
It is also worth noting that the Access OLEDB provider (Microsoft.ACE.OLEDB.12.0) does not exhibit the same problems with dates prior to 1753.
I am trying to write a large dataset (10 cols, 100M records) from R to SAP HANA using RJDBC's dbWritetable in the following way
library("RJDBC")
drv <- JDBC("com.sap.db.jdbc.Driver", "/data/hdbclient/ngdbc.jar", "'")
database <- dbConnect( drv,"jdbc:sap://servername", "USER", "PASS")
dbWriteTable(database, "largeSet", largeSet)
This works, but is extremely slow (75k records per HOUR). I have tested RODBC (sqlsave) as well and this shows the same issue.
Looking at the code behind dbWriteTable it seems that writing is record by record (i.e. the same as insert into) and indeed using a line by line insert into using dbSendUpdate shows the same performance. I have verified that the problem is not in the connection speed itself.
ROracle has a bulk_write option which seems to solve this issue, but since I am trying to write to HANA I need RJDBC or RODBC.
Can anyone tell me how I can speed up the write to HANA by running a bulk write or some other method?
If your main goal is a speed-up, without changing too much else, you could switch to the sjdbc package, which is a lot more performant in this respect than RJDBC (which sadly didn't get much attention in recent years).
While I write this and check back on CRAN, it looks like Simon has just recently found back to it and published a new release only a week ago. This does in fact include an improvement to dbSendUpdate:
https://cran.r-project.org/web/packages/RJDBC/NEWS
I have lots of data to wrangle and I need some help.
I have been using an excel file that has two worksheets of interest to me. They each produce a OLAP pivot table with the data I need to work with. What I would like to do is move those (.odc) connections to access queries so I don't have to hand paste all of this info out and manipulate it and then go through the whole process several more times.
One table is Throughput (number of parts through an operation(s)) by Part Number and by Date. The other is Hours Logged at the operation(s) by Part Number and by Date. I also have a master list of all part numbers with some more data that I have to mix in.
Biggest problem: Each chart is producing its own subset of dates and part numbers so I have to take care to match up the data to run the calculations. I've tried:
By hand. Got tired with that real quick.
Using LOOKUP, VLOOKUP, MATCH with INDIRECT and all sorts of tricks.
It's a mess. But I'm confident that if I can put the original pivot tables into Access I can add a few joins and write up a couple queries and it will turn out beautifully.
Worst comes to worse I can copy/paste the pivot table data into access by hand, but what if I want to change or expand the data set? I'd rather work with the raw data.
EDIT:
The data is held on SQL Server and I cannot change that.
The excel pivot tables use a .ODC file for the connection. They gives the following connection string:
Provider=MSOLAP.3;Integrated Security=SSPI;Persist Security Info=True;Initial Catalog=[MyCatalog];Data Source=[MySource];MDX Compatibility=1;Safety Options=2;MDX Missing Member Mode=Error
Provider=MSOLAP.4;Integrated Security=SSPI;Persist Security Info=True;Initial Catalog=[MyCatalog];Data Source=[MySource];MDX Compatibility=1;Safety Options=2;MDX Missing Member Mode=Error
(I replaced the actual catalog and source)
Can I use the .odc file information to create a pass through query in Access?
Have you consider using a proper OLAP server?
Comparison of OLAP Servers
Once setup you'll be able to connect your Excel's Pivot Table to the server (as well as other reporting tools).
Talked to our IT dept. The guy who built the Cubes is working on querying the same info into MS Access for me.
Thanks everyone.