I am not familiar with Application Engine or PeopleCode but inherited this project when someone left. Seems simple but I'm not sure how to approach it.
I have to load a CSV file that has 5 fields. The last field has multiple values separated by a comma and it is qualified with quotes.
file example:
ID , YEAR, VALUE1 , VALUE2, CODE
87778, 2022, processed, none , 100,40
93332, 2022, processed, none , 60
76633, 2022, error , none , 55,35,9
I have created a File Layout definition and set the qualifier and I can load the file into a staging table but now I want to split the last column (CODE) into individual codes.
I have created 2 PeopleTools Record definitions with a parent/child relationship:
parent Record definition with ID,YEAR,VALUE1,VALUE2, and
child Record definition with ID,YEAR,CODE
I have found that I can use the PeopleCode split function to break the CODE column out into an array containing each value in an element. I'm not sure what the best way to structure the program is though.
Is the staging table necessary?
Or can I use the split function as I read the CSV file in and update the parent/child tables?
Or do I need to keep the staging table and then read out the fields for the parent record and move them to the permanent table and then do the same for the child after using the split function and then loop through the array?
Just looking for some guidance so my first AE project is not a mess.
IMO, there are always multiple ways to achieve the same thing(especially in AE). we choose one based on our requirements and efficiency.
for staging table: In your case, you can ignore the staging table unless you are expecting to load a huge set of data every time or want to do parallel processing. In other words, you can have staging table if you think loading takes a lot of time and you don't want to risk failing that due to other errors.
You can even achieve this whole thing in one peoplecode action without a staging table.
or,
Load the data into staging table and commit.
loop through the data from staging table in AE (having the data in state rec)
Do the transformation as required using peoplecode action
insert data in necessary tables
update status(have a field in staging table) field in staging table, this may come in handy for any analysis/issue in production
Related
I'm trying to get a better understanding of what Rows and Rowsets are used for in PeopleCode? I've read through PeopleBooks and still don't feel like I have a good understanding. I'm looking to get more understanding of these as it pertains to Application Engine programs. Perhaps walking through an example may help. Here are some specific questions I have:
I understand that Rowsets, Row, Record, and Field are used to access component buffer data, but is this still the case for stand alone Application Engine programs run via Process Scheduler?
What would be the need or advantage to using these as opposed to using SQL objects/functions (CreateSQL, SQLExec, etc...)? I often see in AE programs where the CreateRowset object is instantiated and uses a .Fill method with a SQL WHERE Clause and I don't quite understand why a SQL was not used instead.
I've seen in PeopleBooks that a Row object in a component scroll is a row, how does a component scroll relate to the row? I've seen references to rows having different scroll levels, is this just a way of grouping and nesting related data?
After you have instantiated the CreateRowset object, what are typical uses of it in the program afterwards? How would you perform logic (If, Then, Else, etc..) on data retrieved by the rowset, or use it to update data?
I appreciate any insight you can share.
You can still use Rowsets, Rows, Records and fields in stand alone Application Engines. Application Engines do not have component buffer data as they are not running within the context of a component. Therefore to use these items you need to populate them using built-in methods like .fill() on a rowset, or .selectByKey() on a record.
The advantage of using rowsets over SQL is that it makes the CRUD easier. There are built-in methods for selecting, updating, inserting and deleting. Additionally you don't have to worry about making a large number of variables if there were multiple fields like you would with a SQL object. Another advantage is when you do the fill, the data is read into memory, where if you looped through the SQL, the SQL cursor would be open longer. The rowset, row, record and field objects also have a lot of other useful methods such as allowing you to executeEdits (validation) or copy from one rowset\row\record to another.
This question is a bit less clear to me but I'll try and explain. If you have a Page, it would have a level 0 row. It then could have multiple Level 1 rowsets. Under each of those it could have a level 2 rowsets.
Level0
/ \
Level1 Level1
/ \ / \
Level2 Level2 Level2 Level2
If one of your level1 rows had 3 rows, then you would find 3 rows in the Rowset associated with that level1. Not sure I explained this to answer what you need, please clarify if I can provide more info
Typically after I create a rowset, I would loop through it. Access the record on each row, do some processing with it. In the example below, I look through all locked accounts and prefix their description with LOCKED and then updated the database.
.
Local boolean &updateResult;
local integer &i;
local record &lockedAccount;
Local rowset &lockedAccounts;
&lockedAccounts = CreateRowset(RECORD.PSOPRDEFN);
&lockedAccounts.fill("WHERE acctlock = 1");
for &i = 1 to &lockedAccounts.ActiveRowCount
&lockedAccount = &lockedAccounts(&i).PSOPRDEFN;
if left(&lockedAccount.OPRDEFNDESCR.value,6) <> "LOCKED" then
&lockedAccount.OPRDEFNDESCR.value = "LOCKED " | &lockedAccount.OPRDEFNDESCR.value;
&updateResult = &lockedAccount.update();
if not &updateResult then
/* Error handle failed update */
end-if;
end-if;
End-for;
I am trying to utilize a single tabular form to handle a couple minor tables since I cannot have multiple updateable reports on a single page. The easiest way I could figure to adjust what table I was modifying was to use a select list to choose which table I am viewing/editing.
The select list contains a display of "Supervisor" and "School"
resulting in SUPERVISOR and SCHOOL respectively
the items name is P8_TABLE
Region Source:
select *
from #OWNER#."G06_" || :P8_TABLE
The tables are named G06_SUPERVISOR and G06_SCHOOL
However the query returns:
ORA-00933: SQL command not properly ended
I'm afraid that this won't work. As far as I can tell, tabular form is to be based on one table only (i.e. you can't have a JOIN in there; even if you're displaying values from some other table, you'd rather create functions which would return those values).
That being said, principle you'd like to use means that tabular form underlying table is unknown (as it can vary, depending on what you select in P8_TABLE select list item), which - in turn - means that column names also differ (unless all of those tables share the same column list).
If we presume that above is correct, then I'd suggest you to abandon that approach and maintain every "minor table" on its own tabular form page. It will be just a little bit more typing & clicking, but I wouldn't worry about it.
Hopefully, someone else knows how to do it the way you'd want it to.
You cant make a tabular form query source dynamic. And since the two tables you want to be displayed and be updatable doesnt have common number and usage of their columns, the only way I can think of is separating the two tables and diplaying one tabular form for each table. Though, Apex 4.1 or even the latest Apex 5 doesnt allow multiple tabular form, you can make this possible by using javascript's iframe. You'll need to use javascript since you want to modify the interface of the report/page.
I have a Repeat grid layout, as a source is Report definition. The grid displays twenty row per page. So, if there are thirty-three rows, there are four pages.
I have got a task to export all grid's data to CSV. I have found out the pxConvertResultsToCSV activity. It requires to pass PageList with the properties to convert. I use pgRepPgSubSectionMySectionListB.pxResults to do this. But I have realized, that the property pxResults contains only first twenty elements of pgRepPgSubSectionMySectionListB. But I must export to CSV all the rows. How can I reach this? Thank you.
First run your report by calling pxRetrieveReportData activity of class Rule-Obj-Report-Definition in you acticity
Syntex:- call Rule-Obj-Report-Definition.pxRetrieveReportData
It will ask for parameters:-
pyReportName :- your report definition name
pyReportClass:- class of the report definition
pyPageName :- any page name for example ReportListExport. This page must be defined in Pages & Classes of class Code-Pega-List
After successful execution of this step, you will get ReportListExport.pxResults in Clipboard.
Now use this pxResults for export.
There is one more activity to export your Report in excel.
Call pzViewExportToExcel activity after running your report. And keep ReportListExport.pyReportDefinition as step page of this step.
This is preferred one.
This question is a bit old now so I'm sure the OP has probably solved the problem and moved on at this point. But for future viewers there is an easier way to solve this.
Pega includes a gadget called the "Record Editor" which can be used to display a report definition as an editable data table. It shows the provided report definition in a simple table as normal but users can also edit the rows, delete the rows and add new ones. It also includes import and export actions at the top so users can drop the entire resultset being shown in the table to CSV and then re-import changes back in after editing. You can find more information on this gadget and how to use it in this community article
If you simply want to provide an option at the top of a table sourced from a report definition that allows users to export the results as CSV without using the record editor gadget there is an API for that as well. The activity "pxDownloadDataRecordsAsCSV" in class "PegaAccel-Task-DataTableEditor" does this. It accepts the class and name of a report definition as parameters, runs that report and serves up the contents as a CSV file.
The second part here isn't too different from AJ's solution it's just an already existing parameterized activity you can use instead of writing one yourself.
I have a requirement to load the csv into DB using oracle apex or pl/sql code, but the problem is they are asking to load the csv file which will not come with same number of columns and column names .
I should create table & upload data dynamically based on the file name and data that i'm uploading.
For every file i need to create a new table dynamically and insert data that are present in csv file.
For Example:
File1:
col1 col2 col3 col4 (NOTE: If i upload File 1, Table should be created dynamically based on the file name and table should contain same column name and data same as column headers of csv file . )
file 2:
col1 col2 col3 col4 col 5
file 3:
col4 col2 col1 col3
Depending on the columns and file name i need to create table for every file upload.
Can we load like this or not?
If yes, Please help me on this.
Regards,
Sachin.
((Where's the PL/SQL code in this solution!!??! Bear with me... the
answer is buried in here somewhere... I introduced some considerations
and assumptions you will need to think about before going into the
task. In the end, you'll find that Oracle APEX actually has a
built-in solution that satisfies exactly what you've specified... with
some caveats.))
If you are working within the Oracle APEX platform, you will have some advantages. APEX Version 4.2 and higher has a new page element called "Data Loading". The disadvantage however is that the definition of the upload target is fixed and not dynamic. You will need to know how your table is structured prior to loading the data.
One approach to overcome this is to build a generic, two-column table as your target, which will serve for all uploads. Column 1 will be your file-name and column two will be a single clob data type, which will contain the entire data file's contents including the header row. The "Data Loading" element will give the user the opportunity to verify and select this mapping convention in a couple of clicks.
At this point, it's mostly PL/SQL backend work doing the heavy lifting to parse and transform the data uploaded. As far as the dynamic table creation, I have noticed that the Oracle package, DBMS_SQL allows the execution of DDL SQL commands, which could be the route to making custom tables.
Alex Poole's comment is important as well, you will need to make some blanket assumption about the data type or have a provision to give more clues about what kind of data is contained. Assuming you can rely on a sample of existing data values is not good... what if all the values in your upload are null? I recommend perhaps a second column in the data input with a clue about the type of data for each column... just like the intended header names, maybe: AAAAA = for a five character column, # = for a numeric, MM/DD/YYYY = for a date with a specific masking.
The easier route:
You will need to allow your end-user access to a developer-role account on a workspace of your APEX server. It is not as scary as you think. With careful instruction and some simple precautions, I have been able to make this work with even the most non-technical of users. The reason for this is that there is a more powerful upload tool found under the following menu item:
SQL Workshop --> Utilities --> Data Workshop
There is a choice under "Data Load" --> "Spreadsheet Data"
The data load tool will automatically do the following:
Accept a CSV formatted file through a browse function on your client machine
Upload the file and parse the first record for the column layout (names)
Allow the user to create a new table from the uploaded file, or to map to an existing one.
For new tables, each column data type can be declared and also a specific numeric/date mask if additional conversion from the uploaded data is necessary.
Delimiter type, optional enclosures (like double quotes), decimal conventions and currency types can also be declared prior to parsing the uploaded file.
Once the user has identified all these mappings and settings, the table is created with the uploaded data. Any errors in record upload are reported immediately afterwards with detailed feedback on the failed records.
A security consideration to note:
You probably do not want to give end users access to your APEX server's backend... but you CAN create a new workspace... just for your end users... create a new database schema for receiving their uploads, maybe with some careful resource controls. Developer is the minimum role needed... but even if the end users see the other stuff there won't be access to anything important from an isolated workspace.
I have implemented the isolated workspace approach on a 4.0/4.1 release APEX platform a few years back, and it worked nicely. Our end user had control over the staging and quality checking of her data inputs (from excel spreadsheet/csv exports collected from a combination of sources). I suppose it may have been even better to cut her out of the picture entirely and focused on automating the export-review-upload process between our database and her other sources. In this case, the volume of data involved was not great enough (100's to 1000's of records) and the need for manual review and edit of the exported data was very important prior to pushing it into the database... so the human element was still important in this case - it is something you'll want to think about now.
I'm struggling to decide what database schema to use. One large table, or many small (though more difficult to manage).
I have 10 templates each with their own text fields. I am trying to store the text for the templates in a database and then when the web page is called I will show the correct text in the html template. Because a mixture of these templates are to be in a sequence of screens where you can navigate backwards or forwards, I need to be able to sequence them, I can only think of adding a page_number column. I also would like to re-order them and delete them as necessary using the page_number column.
I was planning to do all this in a web application without the need for a standard folder/web page structure, like a small CMS system.
option 1,
I can create one large table with many columns, lot's of which will be empty, over half with each row. Is this bad?
option 2,
I could create many tables using only the relevant template columns required.
The problem I see with this, is the headache of repopulating a column in each table when I delete a row, because I need to re-sequence a column that represents page numbers. Which I reduce if I use one large table.
I've thought of moving page numbers into another table called page_order but I cannot think of a way to maintain an effective relationship between the other tables if I make changes.
I'm yet to figure out how to re-sequence a column in a database when a row is deleted. Surely this is a common problem!?
Thanks for taking the time to help!
Have one table that contains one row per template. It might look like:
id (INT, auto-increment)
page_order (INT, unique key here, so pages cannot have the same number)
field1 (STRING, name of the text field)
value1 (STRING, contents of the text field)
field2
value2
Then you have to decide the maximum fields that any page can have (N) and keep adding field/value columns up to N.
The advantage of this is you have one table that isn't sparsely populated (as long as the templates have about the same number of fields, even if the names of those fields are different).
If you want to make an improvement to his (maybe not necessary for a small amount of data) you could change field to an INT id and connect it to a lookup table that contains (field_id, field_name).