We are pulling an XML report via a web service from a send adapter daily (currently just placing it in a folder via FILE adapter). What is the simplest way to compare new reports/messages against previous reports/message of the same orchestration to get the delta/difference (rather than getting the previous days report from an adapter (FILE, custom DB, etc.)?
Best case scenario for us would be to have the orchestration pull the previous day's message from the message box (is that possible?if so, how?), compare the new message/report to the previous days message/report and create a new report in the same schema but including only the records that have changed (delta).
Related
I’m trying to set up a very simple BAM scenario within BizTalk Server 2013R2 upon which to build, involving tracking just the time of all incoming messages processed by a port.
To this end I have :
Within Excel, created an Activity Definition (called
SimpleReceiveTest) containing a single Item called ReceiveTime of
type milestone (date/time), and a View Definition (also called
SimpleReceiveTest) containing just this Activity Definition and Item.
Imported this BAM definition spreadsheet using bm.exe
Added view rights to SimpleReceiveTest again using bm.exe
Launched the Tracking Profile Editor, imported the BAM Activity
Definition, and mapped ActivityID = MessageID and ReceiveTime =
PortStartTime by drag and drop from the Messaging Property Schema, as
shown below :
Set the Port Mappings for MessageID and PortStartTime to relate to a
test Receive Port ReceivePort1 that I am using for testing. This is
using a pass-through pipeline.
Saved and applied the above Tracking Profile
It is my understanding that for any messages received on port ReceivePort1 I should now get a tracking activity created. However this is not happening – there are no records in any of the BAM tables/views and no data is available within the BAM Portal.
I have tried restarting the hosts, and have verified that the TDDS_FailedTrackingData table is empty, there is nothing relevant in the event log, a Tracking host is running and the SQL Agent Jobs are running. I have also tried running these jobs manually.
Have I missed something, and am I correct in my expectation that this simple scenario should create tracked activities for any messages passing through the Receive Port? If so what can I try to further diagnose this?
Now fixed - it's actually a bug in vanilla BizTalk 2013R2 when using a standard pipeline that has been fixed in CU2.
FIX: BAM tracking doesn’t work when you use the XMLReceive or a custom pipeline in BizTalk Server
I would like to know the Process Integration steps.
Through Outbound ports
If any of the event occurs at AX Dynamics, we just want to know that events in the form of XML(Process Integration).
Example: Sales Order Creation, Customer Creation, Purchase Order Creation..
Outbound ports are only useful for asynchronous communication.
See AX 2012 Export Data with Outbound ports for an example (using the file system).
The steps to initiate sending data is in the AIF_SendCustomer.
As this is no lightweight operation, you may consider logging the records which needs integration in a custom integration table, then doing the processing in batch.
This is done in the insert and/or update and maybe delete method.
Deletes requires you store the RecId field value in the external system to be used for delete requests. The following does not cover this.
For logged table make the following method:
void syncRecord()
{
XXXRecordLog log;
log.RefTableId = this.TableId;
log.RefRecId = this.RecId;
log.insert();
}
Then call this.syncRecord() in the insert and update methods.
In the query to the outbound service be sure to exists join your table and the log table. This way only changed records are exported.
Make a batch job to do the transfer using the AIF_SendCustomer as a template.
After a synchronous (AifSendMode::Sync) transfer of the records, delete the log records (or mark them transferred).
Finally call AIFoutboundProcessingService to flush the file:
new AIFoutboundProcessingService().run();
Try to keeps things simple. It might be simpler to do a comma file export of the changed records!
I want to create a send port that writes all messages going in and out of BizTalk to file.
My organization is using Splunk. Splunk will import data from the file directory to make sense of the various messages.
Is it possible to create filter in a send port that subscribes to "everything"? I could solve this by applying filter for each message type in my system. However, there is a lot of messages going back and forth and I'm wondering if there a simpler solution?
I'm using BizTalk 2013.
Yes, just filter on message type like you said, but rather than selecting = and specifying the message type, just select Exists. That will then match any message that has a message type.
EDIT:
As Johns-305 has pointed out if you have any messages that don't have a message type (e.g. pass through receive locations) you may want to pick BTS.MessageID as that will always exist for a message in the message box.
I have a StoredProcedure that returns a simple table containing several records:
DECLARE #STEPS_TABLE AS TABLE (OrchestrationID uniqueidentifier, [Message] nvarchar(1000));
-- LOADING THE VALUES HERE
SELECT * FROM #STEPS_TABLE As Step FOR XML AUTO, XMLDATA, ELEMENTS
I used the SQL Transport Schema Generation Wizard to create my schema and could configure the port correctly. If I use this schema on my orchestration, it works perfectly. BizTalk starts one instance of the orchestration everytime the #STEPS_TABLE has more than 1 record.
Reading Microsoft technical documentation, they recommend to get several messages in one call and then use the XML pipeline to disassemble the multi-row BizTalk message into a single-row BizTalk message.
I haven't used the XML pipeline before, so I tried the provided steps but couldn't get it to work.
Could somebody provide me a link to a "how to" (didn't find anything until now, after several hours of searching) or give me some hints to succeed.
Thanks in advance.
... some hours later I could figure it out myself. So if anybody comes across the same issue as me, here you have some guidelines to make it work on your environment.
At the end I followed a different walkthrough from Microsoft and avoided the pipeline recommendation altogether. The documentation I found is called "Disassembling Result Sets Using the SQL Adapter" and does exactly what i was looking for. You can just follow the whole walkthrough from Microsoft but avoid the creation of the send port and make some little adjustment on the receive port.
After following the technical document you will end up with two schemas, I will call them message and envelope (contains several messages) for the sake of this excercise. In your orchestration you can create a receiving port that maps to the message and then when you configure it as a SQL Port and you link it to your stored procedure (or select statement), you only have to change the Document Root Element Name to the envelope root name; the XML Receive pipeline (provided by default in BizTalk 2006) will do the magic of disassembling the messages contained in the envelope and instantiating an orchestration for each message.
The Microsoft "Disassembling Result Sets Using the SQL Adapter" walkthrough can be found under:
http://msdn.microsoft.com/en-us/library/aa562098(v=bts.20).aspx
Mission accomplished :)
I have an SQL Database and an ASP.NET website built to put data into the database.
One of the project requirements is to build a system that would let the user upload a Crystal Report to the server and run it as needed. This way, the user could create a customized report (for then turning into management, customers) that wouldn't force them to go through a developer.
I'm looking for suggestions on how to accomplish this goal.
Currently, I'm looking for a way to redirect the database connection in the Crystal Report from the database it was developed with to the database it will eventually run on. However, There doesn't seem to be a simple way to do this.
I'm also investigating the ReportViewer object. However, all the code I have seen involves specifying the query for the report in the code, which isn't acceptable.
One option (which I don't like at all) is to let them write their own queries so they can copy the results into Excel. This would mean a blank textbox and information about the structure of the database. Not a good idea for multiple reasons.
Another option is to create one report for each table (and maybe a few extras), let the user copy the data they want into Excel, and go on their merry way.
tl;dr How do I build a flexible reporting system?
=========================================
Continuation: 08/20/2012
I have decided to go the route of b.pell's extension methods. So far, it has gotten me closer than anything else. My code to bind to the CrystalReportViewer is below:
CrystalReportSource rs = new CrystalReportSource();
rs.Report.FileName = Server.MapPath("ReportFiles/") + Request["reportname"];
string connstring = System.Web.Configuration.WebConfigurationManager.ConnectionStrings["myConnectionString"].ConnectionString;
rs.ReportDocument.ApplyCredentialsFromConnectionString(connstring);
rs.ReportDocument.ApplyNewDatabaseName("myDBName", "mySchemaName");
rs.ReportDocument.Refresh();
CrystalReportViewer1.ReportSource = rs;
This comes very close to working. It works fine on my dev machine, but when I run the code on the server, it gives the following error:
Logon failed.Error in File CrystalReport2 {5D2E82E5-783E-4DFD-A770-C8AE72A51E4E}.rpt:
Unable to connect: incorrect log on parameters. Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: System.Runtime.InteropServices.COMException: Logon failed.Error in File CrystalReport2 {5D2E82E5-783E-4DFD-A770-C8AE72A51E4E}.rpt: Unable to connect: incorrect log on parameters. Source Error: An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.
The error is in this line in the code:
crTable.Location = String.Format("{0}{1}", prefix, crTable.Location.Substring(crTable.Location.LastIndexOf(".") + 1))
When I remove the call to ApplyNewDatabaseName, I am asked to enter the Server Name, the Database name, the Username and the Password or to select Integrated Security. I can't enter the Database Name or the Server Name (those fields are disabled).
Any thoughts?
I think what you're looking for is the Reporting Services, part of Business Intelligence
Or maybe you can setup a UI that let the users pick the tables and columns they need for the report (this way you can limit the information they can access) an write a Dinamic Query Builder Function or something like that.
I answer the changing Crystal Reports connection question a lot (it's something I'd think Crystal would make easier, but I wonder if they don't because that's what their server product does). :D Anyway, you can set the database credentials at runtime. Crystal is very particular in the order it's done, but I have some code that I turned into extension methods that do the trick. This code will go through the main report and all sub reports and change the connection information. This assumes that all sub reports connect to the same database that the main report does (if not, you'll need to modify it to handle multiple connections, but this rarely comes up at least with what I do).
Extension methods to change connection info: http://www.blakepell.com/2012-05-22-crystal-reports-extension-methods
It would be used something like this (although, you're binding to a viewer probably and not exporting, so you could ignore that, this is just for example).
Using rd As New ReportDocument
rd.Load("C:\Temp\CrystalReports\InternalAccountReport.rpt")
rd.ApplyNewServer("serverName or DSN", "databaseUsername", "databasePassword")
rd.ApplyParameters("AccountNumber=038PQRX922;", True)
rd.ExportToDisk(ExportFormatType.PortableDocFormat, "c:\temp\test.pdf")
rd.Close()
End Using
System.Diagnostics.Process.Start("c:\temp\test.pdf")
You could use the Crystal Viewer at this point to deliver the reports and store the report in a database or on the file system (with a db meta data table) and have some predefined connections the user could select from that would be applied when it is run.
You also have the option to write your own front end. In this scenario a user would select a report from your meta data (you could put whatever security on it you wanted, I use AD). Then you can read the report parameters in and lay them out on the web form. When the user fills them in, you then sanatize them and pass them to the report via these extensions and you can output Excel, PDF, Word Doc, RTF, etc. A little more overhead and not the nice preview view, but can work well (I've done something like this in the past). Hope this helps.
About "...let them write their own queries" part of your question.
The solution can be to use some query builder component with friendly user interface which hides from users the complexity of your database and avoid any possible SQL injections.
There are few such products on the market. One of them is called EasyQuery, another one is build by Aspose if I'm not wrong. Try to search in Google for "query bulider for asp.net" or ".net query builder component".