I'm fairly new to U-SQL so this may be a simple question.
I have a field, [utc_timestamp], in an ADL table with a unix time stamp in the form "1497178877" which measures the number of seconds from 1970-01-01.
Is there any easy way to convert this time stamp in U-SQL to both a date in the form of "2017-06-11" and a date time object?
My initial attempt didn't seem to work quite right.
#table =
SELECT * FROM
( VALUES
(1497178877)
) AS T(seconds);
DECLARE #dateStart = new DateTime(1970, 01, 01);
#result =
SELECT #dateStart.AddSeconds(seconds).ToString("yyyy-MM-dd") AS newDateString,
#dateStart.AddSeconds(seconds) AS newDate
FROM #table;
OUTPUT #result
TO "/Temp/Dates/Example1.txt"
USING Outputters.Tsv();
U-SQL is using C# for the expression language, so you can use C#/.NET to do it.
Here is a link answering how to do it in C#: How can I convert a Unix timestamp to DateTime and vice versa?
Since we are currently on .NET Runtime 4.5.2, you will not be able to use the 4.6 built-in method (we plan to upgrade to a newer version of the runtime, but I do not have an ETA yet).
If you want to avoid writing the transformation inline as a C# expression you can either deploy it via VS's code behind, with a U-SQL Func variable or create and register an assembly containing the UDF.
Related
I am using sqlite for local database in mobile and in my database. i want to know that
How to get current date format in SQLITE? I want to get date in the next format: MM/dd/yyyy
To get the current date you can use:
SELECT date('now');
Note: This is NOT a server date, it's the same time you get if you query the date and time directly from your application because SQLITE runs in-process.
It's mostly useful for putting a current time into a table or for some simple calculations if your language's date processing is very poor.
To do the calculations see the SQLITE Documentation
See the docs for formatting too for example:
SELECT strftime('%Y-%m-%d %H:%M:%S', datetime('now'))
According to the SQLite documentation as of this writing (3/30/2020), in the section titled "The DEFAULT clause", it is recommended that a constant value be used instead of a sub-query.
In my experimentation, I also ran into issues with the SQLite CREATE TABLE statement that was generated during model creation by EF Core 3.0 when using SELECT datetime('now'). The SQLite data provider complained of a syntax error when using a SELECT statement within the CREATE table statement. As such, I would recommend using the CURRENT_TIMESTAMP keyword.
for your concrete case, this is what you need:
strftime('%m/%d/%Y',date('now'))
I am trying to create a report that has three optional parameters using a command in Crystal Reports 2008. The only one I can't get to work is the date parameter (seems Oracle does not treat dates the same as SQL Server).
I want the user to be able to pick and choose which parameter they would like to use; one, two, or all three. It works beautifully when I run it in TOAD and hard code the dates, but it will not run in Crystal reports using the syntax to create the parameters.
I have done this a thousand times in a SQL server environment but can't get it to work in Oracle.
The field is type is DATE in the Oracle table.
Here is the syntax from my Report command for the 3 parameters:
AND ( ( CLIL. ITEM_TAG IN ('{?tag}') OR CLS.DESCRIPTION IN( '{?desc}')
OR trunc (CLIL.ISSUE_DATE) BETWEEN to_date ('{?StartDate}', 'mm/dd/yyyy') and to_date ('{?EndDate}', 'mm/dd/yyyy' )))
Crystal reports doesn't accept the Oracle syntax.. you need to use the functions provided by the CR to do manuplations of dates... and then use that in CR.
Create a Start Date and End Date as date parameters in CR and then use those.
(CLIL.ISSUE_DATE) >= {?StartDate} and (CLIL.ISSUE_DATE) < {?EndDate}
Here start date and end date are Date datatype parameters.
if (CLIL.ISSUE_DATE) is a datetime parameter then use the function Cdate provided by the crystal.
I have the following piece of inline SQL that I run from a C# windows service:
UPDATE table_name SET
status_cd = '2',
sdate = CAST('03/28/2011 18:03:40' AS DATETIME),
bat_id = '33acff9b-e2b4-410e-baaf-417656e3c255',
cnt = 1,
attempt_date = CAST('03/28/2011 18:03:40' AS DATETIME)
WHERE id = '1855'
When I run this against a SQL Server database from within the application, I get the following error:
System.Data.SqlClient.SqlException: The conversion of a varchar data type to a datetime data type resulted in an out-of-range value.
The statement has been terminated.
But if I take the piece of SQL and run it from SQL Management Studio, it will run without issue.
Any ideas what may be causing this issue?
Ambiguous date formats are interpreted according to the language of the login. This works
set dateformat mdy
select CAST('03/28/2011 18:03:40' AS DATETIME)
This doesn't
set dateformat dmy
select CAST('03/28/2011 18:03:40' AS DATETIME)
If you use parameterised queries with the correct datatype you avoid these issues. You can also use the unambiguous "unseparated" format yyyyMMdd hh:mm:ss
But if i take the piece of sql and run it from sql management studio, it will run without issue.
If you are at liberty to, change the service account to your own login, which would inherit your language/regional perferences.
The real crux of the issue is:
I use the following to convert -> date.Value.ToString("MM/dd/yyyy HH:mm:ss")
Please start using parameterized queries so that you won't encounter these issues in the future. It is also more robust, predictable and best practice.
I think the best way to work with dates between C# and SQL is, of course, use parametrized queries, and always work with DateTime objects on C# and the ToString() formating options it provides.
You better execute set datetime <format> (here you have the set dateformat explanation on MSDN) before working with dates on SQL Server so you don't get in trouble, like for example set datetime ymd. You only need to do it once per connection because it mantains the format while open, so a good practice would be to do it just after openning the connection to the database.
Then, you can always work with 'yyyy-MM-dd HH:mm:ss:ffff' formats.
To pass the DateTime object to your parametrized query you can use DateTime.ToString('yyyy-MM-dd HH:mm:ss:ffff').
For parsing weird formatted dates on C# you can use DateTime.ParseExact() method, where you have the option to specify exactly what the input format is: DateTime.ParseExact(<some date string>, 'dd/MM-yyyy',CultureInfo.InvariantCulture). Here you have the DateTime.ParseExact() explanation on MSDN)
It's a date format issue. In Ireland the standard date format for the 28th of March would be "28-03-2011", whereas "03/28/2011" is the standard for the USA (among many others).
I know that this solution is a little different from the OP's case, but as you may have been redirected here from searching on google the title of this question, as I did, maybe you're facing the same problem I had.
Sometimes you get this error because your date time is not valid, i.e. your date (in string format) points to a day which exceeds the number of days of that month!
e.g.: CONVERT(Datetime, '2015-06-31') caused me this error, while I was converting a statement from MySql (which didn't argue! and makes the error really harder to catch) to SQL Server.
You could use next function to initialize your DateTime variable:
DATETIMEFROMPARTS ( year, month, day, hour, minute, seconds, milliseconds )
JAVA8: Use LocalDateTime.now().toString()
i faced this issue where i was using SQL it is different from MYSQL
the solution was puting in this format:
=date('m-d-y h:m:s');
rather than
=date('y-m-d h:m:s');
I have an application in asp.net which worked fine untill recently when i changed the datetime format in Regional and Language Settings in Control Panel.
The date time format was default when i installed XP. I chose Indian standard time while installing XP.
I changed the date time format to dd/MM/yyyy HH:mm:ss. And my application started to throw an exception whenever i tried to insert any datetime in to the table.
The exception i get is:
System.Data.SqlClient.SqlException: Error converting data type varchar to datetime.
Please help me on this
Hard to know exactly what's going on without seeing the code that's throwing. However, if you need to communicate dates to SQL Server, it is generally good practice to use the ISO 8601 standard for representation because it is unambiguous and locale-independent. The most important formats are:
yyyy-MM-dd for dates
hh:mm:ss for time
yyyy-MM-dd hh:mm:ss for date/time
My guess is that you have a query that's sending over dates in the current locale, and the locale on the server does not match.
Edit: And for the record, this doesn't preclude anything that Rob said in his answer, i.e. try to avoid passing hard-coded dates or hard-coded SQL at all. This only applies if you need to for some reason.
Edit 2: I've been informed that the yyyy-MM-dd format can still be wrong for some locales; so instead of this, if you need to pass in a literal date string, you should instead use yyyyMMdd.
As per my comment, you'll probably want to make sure you're using code that behaves in a similar way to the code below (i.e. using parameters rather than string concatenation)
var myConnectionString = "connection string goes here";
var myDateValue = DateTime.Now;
using (var connection = new SqlConnection(myConnectionString))
{
using (var command = new SqlCommand("SELECT COUNT(1) FROM dbo.table WHERE datecolumn = #datevalue", connection))
{
var dateValueParameter = new SqlParameter("#datevalue", myDateValue);
command.Parameters.Add(dateValueParameter);
var result = Convert.ToInt32(command.ExecuteScalar());
}
}
Try adding "Current Language=YourLanguage" to the SQL server connection string. Where YourLanguage is the language you want SQL to use when reading values such as the dates.
You can see a list of all languages supported by SQL by executing the following SQL command:
select * from master.dbo.syslanguages
I've found a similar question on stack overflow, but it didn't really answer the question I have. I need to make sure that my asp.net application is formatting the date dd/mm/yyyy the same as my SQL Server 2005.
How do I verify the date culture (if that's what it's called) of the server matches how I've programmed my app? Are there specific database settings and OS settings? Is it table-specific? I don't want to transpose my days and months.
thank you
When you get a DateTime out of the database, it should be in a non-cultured format (like the DateTime object, based on the number of ticks since a certain date). It is only when you are converting that value into a string that you need to be concerned with culture. In those cases, you can use yourDateTimeValue.ToString("dd/MM/yyyy", CultureInfo.InvariantCulture) to make sure that the information displays correctly.
I belive that if you use SqlParameters ADO.NET will take care of the rest and you don't have to worry about it. Besides, it's good for defending against SQL Injection attacks too! :)
** Watch out because SQL DateTime columns are non-nullable and their minimum value is 1/1/1753 while .net DateTimes are non-nullable with min values of 1/1/0001. **
If you're pulling data from a real DateTime column, by default it will always be in the same standard format. For saving the data to the column, you might want to specify the SqlDbType.DateTime in your parameter.
i ripped this off of http://bytes.com/forum/thread767920.html :
com.Parameters.Add("#adate", SqlDbType.DateTime).Value = DateTime.Now;
Well, if you keep datetime fields in the DB you shouldn't worry about it.
As long as you keep the dates in app strongly typed (DateTime variables) and send the dates through prepared statements with DBParameter/SqlParameter your DB will take them as is.
If you use strings to hold your dates in code, some casts will ensure you send the right values:
string sqlCmd = #"SELECT *
FROM MyTable
WHERE MyDateField = CONVERT(datetime, '{0}', 101)";
// assuming myDateString is a string with a date in the local format
sqlCmd = string.Format(sqlCmd,
Convert.ToDateTime(myDateString).ToString("yyyyMMdd"));
(the code is ugly, but hopefully it gets the point across)
As others have mentioned, you should be OK as far as storing datetimes culturally. What I would recommend is that you store all of your times as standard UTC time. In SQL Server 2005 and older there is no way to store time zone information, but if everything is stored in universal time, you should be OK because the time can be converted to the local time later on.
SQL Server 2008 does have some datatypes that are aware of time zones, and if you're using .NET 3.5 there are tools to assist with time zone handling/conversions.
Definitely keep times in universal format. This will make a world of a difference if you have to work in multiple time zones.