We have a PeopleSoft installation and I am building a separate web application that needs to pull data from the PeopleSoft database. The web application will be on a different server than PeopleSoft, but the same internal network.
What are my options?
This one's an oldie but it may still be of interest.
PeopleSoft has it's own schema within the host database (Oracle, SQL Server, DB2 etc) which are the PSxxx tables, eg: PSRECDEFN is the equivalent of Oracle's DBA_TABLES. These tables should not be touched by any external code. The application tables are stored in PS_xxx tables, eg: PS_JOB. These tables can be read and updated by any SQL code.
Many batch programs in PeopleSoft (eg: Application Engines, COBOL or SQRs) access the tables directly, and this is the fastest way to get data into or out of the database. However PeopleSoft has quite a rich application layer which is bypassed when doing direct SQL. This application layer must be replicated in direct SQL code, especially for inserts or updates. There may be updates to other tables, calculations or increments of database-stored counters.
To determine how to do this one must look through the PeopleCode (a VB6-like interpreted language), page design (via Application Designer) and use the PeopleCode and SQL trace tools. These days the application layer is huge, so this can be a lengthy task for non-trivial pages. PeopleSoft groups related pages into "Components", and all pages in the component are saved at the same time.
Component Interfaces were introduced with PeopleTools 8 as a means to avoid doing all of this. Using a generator within the PeopleSoft app designer, a Component Interface is generated based on the component. For many components these can be used to access the pages as a user would, and can be accessed via PeopleCode programs, and therefore via App Engine programs and via the Integration Broker. They can also be wrapped in Java code and access directly by code able to execute against the app server with a web service wrapper. This method is best for low-volume transactions: heavy extracts work better with native SQL.
The online development and tracing tools in PeopleSoft are pretty good, and the documentation is excellent (although quite extensive) and available on: http://download.oracle.com/docs/cd/E17566_01/epm91pbr0/eng/psbooks/psft_homepage.htm
If you are just looking at bringing out data from a given Component, the easiest way would be to turn on the SQL trace (under the utilities menu in PeopleSoft) and bring up some records for the Component. Wading through the trace file will give you a good idea of what to do, and much of the SQL could be cut and pasted. Another method would be to find an existing report that is similiar to what you are trying to do and cut out the SQL.
Have a PeopleSoft business analyst on hand to help you develop the requirements wouldn't hurt either.
Yes - Integration Broker is Peoplesoft's proprietary implementation of a publish/subscribe mechanism, speaking xml. You could of course just write code that goes against your database using JDBC or OLE/ODBC. Nothing keeps you from doing this. However, you must understand the Peoplesoft database schema, so that you are pulling from, or inserting/updating/deleting all of the proper data. Peoplesoft takes care of this for you.
Also, check out Component Interfaces - and they are exposed as an API to Java or C/C++.
I guess it depends on your requirement, and which version of PeopleSoft you're on.
Do you want real-time lookup? If that's the case then you'll want to look at Web Services/Integration Broker.
If you want a batch/bulk export then a scheduled App Engine would do the trick.
The best way is to use Integration Broker (IB) services to expose the PeopleSoft database data to external applications. The external application will be able to access the PeopleSoft IB services as XML over HTTP, thus allowing you to use any widely used XML parsers for this purpose.
The problem with component interfaces as opposed to Integration Broker is that component interfaces tend to be much slower than direct DB access from within IB service PeopleCode. Also future additions to the component attached to the component interface sometimes tend to 'break' the interface.
For more details on PeopleSoft Integration broker, you can access the online documentation at http://docs.oracle.com/cd/E26239_01/pt851h3/eng/psbooks/tibr/book.htm
Going directly to the database means you have to re-create the presentation logic... see my longer answer above. You can do this for simple pages but otherwise using a component interface is the way to go.
You can also write a sqr process for bulk data extraction. SQR will create the output file which the other application can pick. SQR would be faster than the application engine programs as it performs most of the operations in memory.
Related
I would like to create and archive a collection of demo ASP.NET web form applications that show projects with certain features in the sense "this feature can be implemented like this" -- to be presented to a potential customer.
Before the presentation, I would like to get the selected set of demo and install them easily to the notebook. Each of the demos will be "frozen". The target notebook is not the customer's one. It is one of our ones that is bring to the customer for the presentation. This way, it can be prepared in the sense that a named MS SQL instance with the fixed name can be ready, etc.
Can you share some experience with such situation? (I do not want to have marked this question as of opinionated; so please, if you have some explicit links to the related documents or explicit suggestions...)
Here are some other facts and initial ideas:
Each of the demo projects uses two databases: xxx_users (the standard ASP.NET authentication...), and xxx_application (and possibly xxx_external) where xxx is a prefix for the specific project.
The demo application is expected to be compiled (binary only, no sources needed for the presentation).
The Web.config files can use the local\SQLINSTANCEFORDEMOS in connection strings.
The SQL instance has a fixed name, fixed administrator account (like sa) and fixed password for the logging to the SQL instance. This way, it can be included in the Web.config files.
The sample data can be fairly big (not extremely tiny).
The application will use its own SQL tables in the xxx_application database.
The application will simulate the outer database that is accessed from the web application can be simulated by xxx_external database.
This way, I should be able to create and archive SQL backups of xxx_users, xxx_application, and xxx_external databases, plus the archive of the web app binary.
Have you ever encountered this situation? Is the approach reasonable? Could you share some better ideas?
Team A has an enterprise app that uses ADO.NET for data access that executes stored procedures. The data access is encapsulated in it's own project (let's call it DAL.dll)
Team B is creating another unrelated app that's reusing the stored procedures in the enterprise app. This app is currently using the MS application block for data access. The issue we run into is that whenever Team A make any change to the input/output params in the stored procedures, there is a runtime error in Team B's app and this app needs to be updated to accommodate the additional params (or params that were removed). So, most of these go unnoticed until a user complains. At the very least, we would like to have the app throw a compilation error so that the build process warns us of the changes made.
One way to do this is to have Team B's project add a reference to the DAL.dll
I'd like to know if there are any other cleaner ways of solving the issue. We are ready to replace Team B's MS Data application block to use a different technology (Entity Framework?) if necessary.
Among the other answers, I'd strongly suggest getting those stored procedures into source control, in a Database Project. You then may be able to use the features of your source control system to do several things:
Lock some of the code so that it cannot be changed
Give you notifications if the code is changed
Warn you if the stored procedures change in a way that would prevent them from being called
Branch the stored procedures so that each team can have their own version of changed code, while keeping the unchanged stored procedures common. You of course will need to separate the different versions in the database.
I agree with the other posters on this thread that you should not share stored procedure's across different .NET DLL's, that is just a recipe for disaster. I would also shy away from ORM's like Entity Framework if you are doing anything at all complicated with your database schema because ORM's excel at getting a simple object model translated from your .NET application classes into SQL tables and SP's, but traditionally do poorly at optimizing them for performance on the database side. There will be people who claim otherwise, and they may have a valid point if you are an expert in wrangling an ORM to do waht you want like they are, but chances are you are not and it will cause you headaches in the long run.
A shared data access layer might work, but conceptually you are then just changing the implementation of the dependency from some code that a DBA wrote to some code that a .NET programmer wrote. Yes, you can use integration tests to achieve better verifiability, but the same case could be made for SQL with tools like Red Gate's SQL Test. I would shy away from this approach if the two applications are already experiencing some sort of pain from sharing SP's. That is an indication that the dependency just should be done away with.
If it were up to me, I'd just make a new schema for Team B's app. You can read more about schemas in SQL Server here: MSDN Schema description for 2008 R2. You can think of them as namespaces for SQL Server but with some additional bells and whistles like permission and access control. Separating out your different applications into separate schemas on the same shared database will probably make for the most flexible implementation in the long run.
unrelated app that's reusing the stored procedures in the enterprise app
If these two application are really unrelated why are those sharing procedures or even the same database. I know this is a long read, but I recommend you to read this: A Better Path to Enterprise Architectures
The partioning concept in there relates to the bounded context in Domain driven design:
Multiple models are in play on any large project. Yet when code based on distinct models is combined, software becomes buggy, unreliable, and difficult to understand. Communication among team members becomes confusing. It is often unclear in what context a model should not be applied.
Therefore: Explicitly define the context within which a model applies. Explicitly set boundaries in terms of team organization, usage within specific parts of the application, and physical manifestations such as code bases and database schemas. Keep the model strictly consistent within these bounds, but don’t be distracted or confused by issues outside.
It is expected you end with problems when you don't explicitely deal with this. You're lucky you're seeing early failures, as it can turn into problems much harder to find on the long run.
Analyze the problem again with the above in mind. Consider if you're missing some explicit context where this common functionality should live.
My question is: which team owns the store procedured and the database shared? Usually as a good architecture/design, you should not have two different apps sharing same database / procedures.
A better way to share data/functionality between two different applications is through a services or API, so the team who owns the functionality would be responsible to maintain it.
Also, have a good communication between both teams is highly recommend.
Depending on the owner of the DAL project, you could host web services and share the API. That way, you separate the Data Access Layer from the business logic, which allows anyone to use the same DAL without having to publish it to each different location.
From my point of view, it looks like both Team A and Team B should share the same core model and look at Multitier architecture as a possible solution.
It sounds like it would make sense to create a shared DAL that both applications can share.
I would add unit tests (or really integration tests) to make sure the DAL is compatible with the apps after changes. That way your tests would fail if incompatible changes have been made
"I'd like to know if there are any other cleaner ways of solving the issue."
The cleanest way is for Team B to sit down with Team A and encapsulate the relevant business logic into a shared API. It doesn't matter so much how you implement that API; what does matter is that the API's interface is documented and versioned so everyone knows what to expect.
One reasonable mechanism for this in a .NET environment is to use Microsoft's WebAPI.
In short, the question of "how do we share a stored procedure?" is most likely looking at the wrong level of abstraction.
I'd like to know the best architecture.
We have a web application running different domains. Each domain has its own MySQL database but the db structure is the same for all of them.
We have a web application for the visible part of the application.
We have a dataLogic project
We have a dataEntities project
We have a dataAccess that contains only the methods to connect to the data base.
Before we called stored procedures on a database. But we had to change it because the performance was bad. Also, the problem was that every change we made we had in a stored procedure we had to copy to every database.
We are thinking in using a WebService to retrieve the data. Every domain can call the web service with a connection string and connect its database to retrieve data. This way when we change a SQL query we only have to compile the webService and change it, we don't have to change versions on multiples domains.
Also, what do you think about the SQL queries? Since we don't want to keep using stored procedures, what is the best way to do it? Directly from code?
Thanks
T
If you have multiple Database servers you will have to make Structural changes from one DB to another one way or another. There are many tools to change Database structures. These tools will look for differences between Schema, and will either generate the SQL code for you, or do the changes by itself (it depends a lot in the tool, there are powerful ones and not so powerful ones). Please do take a look at Toad for MySql. Now, for the Data changes, you may want to replicate the data from one Database to another. This is done through Replication.
We are thinking in using a WebService to retrieve the data. Every
domain can call the web service with a connection string and connect
its database to retrieve data.
This sounds like a good idea and since you already have "dataAccess" and "dataLogic" projects, it should not be too hard to make the services.
Also, what do you think about the SQL queries? Since we don't want to
keep using stored procedures, what is the best way to do it? Directly
from code?
I don't think it is a good practice to have the SQL queries directly into your code, but it depends in a lot of things, so I would suggest Stored Procedure vs Hard-Coding the queries, or LinQ (Entity Framework 4.1).
Good luck with your project and I will take a look at this thread frequently to see what you end up doing.
Have fun!
Hanlet
my next assignments is to build 2 information portals for customers. These portals will be login protected sites and contain a set of pages displaying information like orders, invoices, pdf-files ... for the authenticated user (all presented as lists with links to detail pages). The users and the data are stored in an Oracle database. The portals differ in some of the features and in the layout.
My standard approach is to build an individual ASP.net Web Application for every portal.
But this is not the best way to get something reusable. So for these two projects my idea is to create a set of WCF services to get the Data from the Oracle database and to build user controls to display the different elements in Umbraco. This way I hope to get a set of independent, reusable “modules” which can be used to build these portals.
Now my question: is Umbraco a good platform for this type of projects? And is my “concept” a valid approach?
Kind regards
Volkmar
Umbracois very flexible. ON the one hand there is the question about security: With Umbraco you can use any Membership Provider you want for all visitors ( also with member roles).
On the other hand you have the question of the integration: With Umbraco you can create usercontrols, xslts or razor files as macros (which can be seen as the reusable modules).
For Xslt you can implement your own XsltExtension which pulls the external content as XPathNodeIterator you can use in every Xslt macro. For ascx files or razor you can use LinQ2Umbraco, your own objects etc to connect to the oracle database.
You also can use some sort of caching functionality to reduce the db-calls. On the other hand is one of the biggest advantages that Umbraco stores all the content as xml and object tree in memmory. So it is very fast in content rendering. With every database call you are loosing a little bit of this advantage.
hth, Thomas
Ruben Verbourgh began the Oracle4Umbraco project to create an abstracted fork for the Datalayer to support running on an Oracle DB. You can find it at http://oracle4umbraco.codeplex.com/, although it has no active releases, so build from source and YMMV.
Volkmar, your concept is perfectly sound - although you might want to consider using the Umbraco data store as the persistence layer for your data rather than in the Oracle DB itself. You get XML content versioning, caching, and all the benefits of the content-management side of things, in a robust and flexible framework which you can expose to other apps later should you so need to, through the Umbraco APIs and web services.
HTH,
Benjamin
content management of website becomes simplified with Umbraco.
But if you are planning to use Oracle as backend, Umbraco does not have support for it.
So decide carefully as to what parameters can be compromised.
Good luck.
My friend asked me to make a registration software for his blood donation camp. He told me that he can't host an ASP.NET or PHP page.
Also that since they'll be running the software in many systems, which may not be interconnected, they may not be able to use a Database server.
Is there a way to store/read data from an MS-Excel or MS-Access file instead of any other database?
Also that I wish to make the front end using HTML & JavaScript/JQuery.
As you problem is little bit twicky. Since your app will run on various machine so you want to develop standalone app in Html / Jquery. Also you cannot use db server bcoz of lack of connectivity. So in such scenario i will suggest you to go for Win Form app with a lightweight db like MS Access and Sqlite. Which ever you prefer. Since you are keeping Network aside so i suggest you to not use HTML/Jquery to use access or Excel as it looks logical very easy to use interop with excel but implementation is quite messy. In that time you can finalize the winform app with any of the database with lots of ease.