I want to trigger an SQL job which runs an SSIS package from an AX Job, I successfully ran SQL code fetching some records from an SQL table by creating a menu item for the job and have it run on the server instead of the client but the following code runs without errors but the Job's not started?
CODE:
static void TriggerAllocation(Args _args)
{
UserConnection userConnection;
Statement statement;
str sqlStatement;
SqlSystem sqlSystem;
SqlStatementExecutePermission sqlPermission;
;
sqlSystem = new SqlSystem();
sqlStatement = "EXEC MSDB.dbo.sp_start_job #Job_Name = 'MyJob'";
userConnection = new UserConnection();
statement = userConnection.createStatement();
sqlPermission = new SqlStatementExecutePermission(
sqlStatement);
sqlPermission.assert();
statement.executeQuery(sqlStatement);
CodeAccessPermission::revertAssert();
I can't find any more clues in e.g. eventviewer, SQL logs as for what went wrong..
Kind regards,
Mike
[UPDATE] Thanks to Alex K I solved it!
using
statement.executeUpdate(sqlStatement);
instead of
statement.executeQuery(sqlStatement);
did the trick!
Keep in mind that running the job directly from AX won't work despite: server static void
You'll have to create an Menu Item of type action with property RunOn=Server
I should have given my comment as an answer but wasn't paying attention:
Try server static void Trigger... and perhaps statement.executeUpdate(...) instead of executeQuery
Calling EXEC from executeQuery is not supported.
See this question:
How to get the results of a direct SQL call to a stored procedure?
Related
I use airflow python operators to execute sql queries against a redshift/postgres database. In order to debug, I'd like the DAG to return the results of the sql execution, similar to what you would see if executing locally in a console:
I'm using psycop2 to create a connection/cursor and execute the sql. Having this logged would be extremely helpful to confirm the parsed parameterized sql, and confirm that data was actually inserted (I have painfully experiences issues where differences in environments caused unexpected behavior)
I do not have deep knowledge of airflow or the low level workings of the python DBAPI, but the pscyopg2 documentation does seem to refer to some methods and connection configurations that may allow this.
I find it very perplexing that this is difficult to do, as I'd imagine it would be a primary use case of running ETLs on this platform. I've heard suggestions to simply create additional tasks that query the table before and after, but this seems clunky and ineffective.
Could anyone please explain how this may be possible, and if not, explain why? Alternate methods of achieving similar results welcome. Thanks!
So far I have tried the connection.status_message() method, but it only seems to return the first line of the sql and not the results. I have also attempted to create a logging cursor, which produces the sql, but not the console results
import logging
import psycopg2 as pg
from psycopg2.extras import LoggingConnection
conn = pg.connect(
connection_factory=LoggingConnection,
...
)
conn.autocommit = True
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
logger.addHandler(logging.StreamHandler(sys.stdout))
conn.initialize(logger)
cur = conn.cursor()
sql = """
INSERT INTO mytable (
SELECT *
FROM other_table
);
"""
cur.execute(sql)
I'd like the logger to return something like:
sql> INSERT INTO mytable (
SELECT ...
[2019-07-25 23:00:54] 912 rows affected in 4 s 442 ms
Let's assume you are writing an operator that uses postgres hook to do something in sql.
Anything printed inside an operator is logged.
So, if you want to log the statement, just print the statement in your operator.
print(sql)
If you want to log the result, fetch the result and print the result.
E.g.
result = cur.fetchall()
for row in result:
print(row)
Alternatively you can use self.log.info in place of print, where self refers to the operator instance.
Ok, so after some trial and error I've found a method that works for my setup and objective. To recap, my goal is to run ETL's via python scripts, orchestrated in Airflow. Referring to the documentation for statusmessage:
Read-only attribute containing the message returned by the last command:
The key is to manage logging in context with transactions executed on the server. In order for me to do this, I had to specifically set con.autocommit = False, and wrap SQL blocks with BEGIN TRANSACTION; and END TRANSACTION;. If you insert cur.statusmessage directly following a statement that deletes or inserts, you will get a response such as 'INSERT 0 92380'.
This still isn't as verbose as I would prefer, but it is a much better than nothing, and is very useful for troubleshooting ETL issues within Airflow logs.
Side notes:
- When autocommit is set to False, you must explicitly commit transactions.
- It may not be necessary to state transaction begin/end in your SQL. It may depend on your DB version.
con = psy.connect(...)
con.autocommit = False
cur = con.cursor()
try:
cur.execute([some_sql])
logging.info(f"Cursor statusmessage: {cur.statusmessage})
except:
con.rollback()
finally:
con.close()
There is some buried functionality within psycopg2 that I'm sure can be utilized, but the documentation is pretty thin and there are no clear examples. If anyone has suggestions on how to utilize things such as logobjects, or returning join PID to somehow retrieve additional information.
How to set up the SQL Server database for Semantic logging.
Does the table for logging information needs to be created earlier?
If yes, what is the schema to be used.
I have the following code :
var listener = new ObservableEventListener();
string connectionString = #"Data Source=nibc2025;Initial Catalog=TreeDataBase;Integrated Security=True;User Id=sa;Password=nous#123";
listener.EnableEvents(AuditingEventSource.Log, EventLevel.LogAlways, Keywords.All);
databaseSubscription = listener.LogToSqlDatabase
(
"Test",
connectionString,
"Traces",
Buffering.DefaultBufferingInterval,
1,
Timeout.InfiniteTimeSpan, 500
);
// The following one line of code is not part of this function.
// It is just added here to show this is how I log my information.
// Inside LogInformation method I call the 'Write' method
AuditingEventSource.Log.LogInformation("sgsgg", "sgsg");
databaseSubscription.Sink.FlushAsync().Wait();
Well, Since this thread gets the most hits on Google regarding Semantic Logging onto SQL DB or even SLAB ...
The scripts to create the DB lies here
\packages\EnterpriseLibrary.SemanticLogging.Database.1.0.1304.0\scripts
And to create the EventSource and Fire the Blocks, information are given here, for a shortcut and quick fix solution
http://entlib.codeplex.com/discussions/540723
Regards,
OK. I got it. the script was in the packages folder. I had overlooked that.
I am working with MySQL in my .net web app with the help of MySQL connector 5.0.9.0.
I am getting data from a web service and inserting it in local database. I only want to insert unique rows in my local database. I came to know that when an duplication occurs, MySQL returned an error code: 1062.
All i want to know that how i could i get this code in a variable and check it against my condition like:
if(errorCode == "1062")
{
Response.Write("Record already exists");
}
else
{
// add the record to database
}
Thanx in advance
Have a look at MySqlException class members. There is a Number property - Gets a number that identifies the type of error.
MySqlException Class.
Please see below line of code for assistance.
private static readonly dynamic db= Database.OpenNamedConnection("DefaultConnectionString");
var idRecord = db.TableName.FindAll(db.TableName.phoneNo == phoneNo && db.TableName.password == pass).FirstOrDefault();
My question is, how can I see the value of db.TableName.phoneNo during Debug.
Any help is welcome.
Thanks.
Liaqat
The only way to debug this is to have the query output to the console. Then you can run it in your database. For more information, see this question. How to log generated sql queries out of Simple.Data ORM for .NET
I am trying to determine if I have a database connection leak.
So I need to see the number of open connections.
I have some simple test code that creates a leak:
protected void Page_Load(object sender, EventArgs e)
{
for(int i = 0; i < 100; i++)
{
SqlConnection sql = new SqlConnection(#"Data Source=.\SQLExpress;UID=sa;PWD=fjg^%kls;Initial Catalog=ABC");
sql.Open();
}
}
Note there is no .Close and this does infact crash after being run 3 times in quick succession.
In order to measure the leak I am running the Performance monitor and measuring SQLServer: General Statistics/User Connections:
(source: yart.com.au)
However, these seem to be zero when I run my code:
(source: yart.com.au)
What should I change to actually see the connections?
ANSWER
I have approved an answer below. Even though it doesn't use the performance tools, its good enough for my use. Bottom line is I wanted to see how many connections remain open after opening a web page and this did the trick.
You can try running a query against the master db like this:
SELECT SPID,
STATUS,
PROGRAM_NAME,
LOGINAME=RTRIM(LOGINAME),
HOSTNAME,
CMD
FROM MASTER.DBO.SYSPROCESSES
WHERE DB_NAME(DBID) = 'TEST' AND DBID != 0
See this link for more details.
Have you tried running the sp_who stored proc? If there are stale open connections they should show up there.
To show just the sa users processes run:
EXEC sp_who 'sa'