BLOB to CLOB Conversion : Cyrillic characters are not getting converted properly
Based on our product migration, I have an requirement to replace below mentioned single byte character to Double byte character in each and every record but the respective column is BLOB datatype encoded with AL32UTF8 so I'm using the below procedure to convert column from BLOB to CLOB and replace the single byte character to Double byte character and convert back to CLOB to BLOB.
NLS_CHAR--> AL32UTF8
For ex:
when I tried with CL8ISO8859P5 character set in CLOB conversion function, Bulgarian data is corrupted and If I tried with AL32UTF8 character set then the respective Cyrillic character got corrupted.
select convert_to_clob(XMLRECORD) From F_TEC_OUTPUT_T
CLOB conversion with AL32UTF8 characterset:
ONE�АНУЛИРАНЕ НА ОПРОСТЕНА ФАКТУРА_�THREE
CLOB conversion with CL8ISO8859P5 characterset:
ONEўаааЃааа ааа аа ааа ааЁаЂааа аЄаааЂаЃа а_ўTHREE
select recid, dbms_lob.substr(XMLRECORD) as raw_xrec
from F_TEC_OUTPUT_T
[enter image description here][1]
Please suggest the proper way to convert this.
Single byte ---> Double byte
ў --> яЃО
§ --> яЃН
ќ --> яЃМ
In RAW:
FE --> EFA3BE
FD --> EFA3BD
FC --> EFA3BC
FB --> EFA3BB
l_clob CLOB;
l_dest_offset NUMBER := 1;
l_src_offset NUMBER := 1;
l_lang_context NUMBER := dbms_lob.default_lang_ctx;
l_warning NUMBER;
BEGIN
dbms_lob.createtemporary(l_clob, TRUE);
dbms_lob.converttoclob(dest_lob => l_clob,
src_blob => l_blob,
amount => dbms_lob.lobmaxsize,
dest_offset => l_dest_offset,
src_offset => l_src_offset,
blob_csid => nls_charset_id('CL8ISO8859P5'),
lang_context => l_lang_context,
warning => l_warning);
RETURN l_clob;
END convert_to_clob;
CREATE OR REPLACE FUNCTION convert_to_blob(l_clob CLOB) RETURN BLOB IS
l_blob BLOB;
l_dest_offset NUMBER := 1;
l_src_offset NUMBER := 1;
l_lang_context NUMBER := dbms_lob.default_lang_ctx;
l_warning NUMBER;
BEGIN
dbms_lob.createtemporary(l_blob, TRUE);
dbms_lob.converttoblob(dest_lob => l_blob,
src_clob => l_clob,
amount => dbms_lob.lobmaxsize,
dest_offset => l_dest_offset,
src_offset => l_src_offset,
blob_csid => nls_charset_id('AL32UTF8'),
lang_context => l_lang_context,
warning => l_warning);
RETURN l_blob;
END convert_to_blob;
CREATE OR REPLACE PROCEDURE convert_blob_file(p_tname in varchar2,parallel_no_cnt in number default 1,record_cmt_cnt in number default 1) IS
TYPE cur_typ IS REF CURSOR;
getcursordata cur_typ;
ConversionRecID VARCHAR2(2000);
BinaryValueSource BLOB;
BinaryValueDest BLOB;
BinaryValueTemp CLOB;
cnt NUMBER := 1;
Dicttablename VARCHAR2(2000);
Marker_R_count NUMBER := 0;
XmlrecordTemp CLOB;
Selectstring VARCHAR2(2000);
select_table_query VARCHAR2(1000):= 'SELECT /*+ PARALLEL('''||parallel_no_cnt||''') */ RECID,XMLRECORD FROM ' || p_tname || ' WHERE XMLRECORD IS NOT NULL';
BEGIN
OPEN getcursordata FOR select_table_query;
LOOP
FETCH getcursordata INTO ConversionRecID, BinaryValueSource;
EXIT WHEN getcursordata%NOTFOUND;
BinaryValueTemp := convert_to_clob(BinaryValueSource);
BinaryValueTemp := REPLACE(BinaryValueTemp,CHR(53662),CHR(15705022)); -- #FM Separator
BinaryValueTemp := REPLACE(BinaryValueTemp,CHR(49831),CHR(15705021)); -- #VM Separator
BinaryValueTemp := REPLACE(BinaryValueTemp,CHR(53660),CHR(15705020)); -- #SM Separator
BinaryValueTemp := REPLACE(BinaryValueTemp,CHR(53659),CHR(15705019)); -- #TM Separator
BinaryValueDest := convert_to_blob(BinaryValueTemp);
EXECUTE IMMEDIATE 'UPDATE /*+ PARALLEL('''||parallel_no_cnt||''') */'||p_tname||' SET XMLRECORD = :v1 WHERE RECID= :v2' USING BinaryValueDest,ConversionRecID;
IF cnt=record_cmt_cnt THEN
cnt:=1;
COMMIT;
ELSE
cnt := cnt + 1;
END IF;
END LOOP;
CLOSE getcursordata;
DBMS_OUTPUT.PUT_LINE('Time------'||to_char(sysdate, 'HH24:MI:SS'));
END;```
[1]: https://i.stack.imgur.com/t9w19.jpg
Related
i wrote a procedure that deals with writing the contents of the QUERY column of ALL_MVIEWS to a file:
DECLARE
v_out_dir_name VARCHAR2(30) := 'DIR_TEST';
v_out_dir_path VARCHAR2(60);
v_count_object_elab NUMBER := 0;
CURSOR c_mviews IS
SELECT
LOWER(MVIEW_NAME) || '.sql' AS FILE_NAME
, QUERY AS SCRIPT
FROM ALL_MVIEWS
;
v_file UTL_FILE.file_type;
BEGIN
FOR r_mview IN c_mviews LOOP
v_file := UTL_FILE.fopen (v_out_dir_name, r_mview.FILE_NAME, 'w');
UTL_FILE.putf (v_file, r_mview.SCRIPT);
UTL_FILE.fclose (v_file);
v_count_object_elab := v_count_object_elab + 1;
END LOOP;
IF v_count_object_elab = 0
THEN
DBMS_OUTPUT.PUT_LINE('NESSUN FILE ELABORATO');
END IF;
EXCEPTION
WHEN OTHERS THEN
DBMS_OUTPUT.PUT_LINE('ERRORE = ' || SQLERRM);
IF UTL_FILE.IS_OPEN (v_file) THEN
UTL_FILE.FCLOSE (v_file);
END IF;
RAISE;
END;
/
But the "FOR r_mview IN c_mviews LOOP" statement generates the following error:
Report error -
ORA-06502: PL/SQL: errore di numero o valore
ORA-06512: a line 35
ORA-06512: a line 16
ORA-06512: a line 16
06502. 00000 - "PL/SQL: numeric or value error%s"
*Cause: An arithmetic, numeric, string, conversion, or constraint error
occurred. For example, this error occurs if an attempt is made to
assign the value NULL to a variable declared NOT NULL, or if an
attempt is made to assign an integer larger than 99 to a variable
declared NUMBER(2).
*Action: Change the data, how it is manipulated, or how it is declared so
that values do not violate constraints.
The error is thrown on a materialized view that has QUERY_LEN = 39000.
How can I solve the problem?
Many thanks in advance.
I don't think you are using UTL_FILE correctly. The putf() procedure is for writing and formatting (using printf() style). Try using just put() instead. I did run your script on my database and it executed fine with putf() and put(), but put() is more appropriate.
Also, beware that QUERY is a LONG. If you have a query over 32k I think you will get the error you are seeing. There are some great write-ups online about how awful LONGs are to work with.
I think easiest thing to do is convert the LONG to CLOB with a CREATE TABLE, read that table and then drop it. Here is a routine to do just that. It will create a copy of all_mviews and convert the query to a CLOB and write that in chunks to a file per view.
DECLARE
v_out_dir_name VARCHAR2(30) := 'MVIEW';
v_count_object_elab NUMBER := 0;
v_cursor SYS_REFCURSOR;
v_file utl_file.file_type;
v_start INTEGER;
v_buffer VARCHAR2(1024);
v_file_name VARCHAR2(1024);
v_query CLOB;
table_or_view_does_not_exist EXCEPTION;
PRAGMA EXCEPTION_INIT(table_or_view_does_not_exist,
-00942);
BEGIN
BEGIN
EXECUTE IMMEDIATE 'DROP TABLE all_mviews_clob';
EXCEPTION
WHEN table_or_view_does_not_exist THEN
NULL;
END;
EXECUTE IMMEDIATE 'CREATE TABLE all_mviews_clob AS SELECT mview_name, to_lob(query) AS query FROM all_mviews';
OPEN v_cursor FOR q'[SELECT lower(mview_name) || '.sql' AS file_name,
query AS script
FROM all_mviews_clob]';
LOOP
FETCH v_cursor
INTO v_file_name,
v_query;
EXIT WHEN v_cursor%NOTFOUND;
v_file := utl_file.fopen(location => v_out_dir_name,
filename => v_file_name,
open_mode => 'w');
v_start := 1;
FOR i IN 1 .. ceil(dbms_lob.getlength(lob_loc => v_query) / 1024)
LOOP
v_buffer := dbms_lob.substr(lob_loc => v_query,
amount => 1024,
offset => v_start);
IF v_buffer IS NOT NULL THEN
utl_file.put(file => v_file,
buffer => v_buffer);
utl_file.fflush(file => v_file);
END IF;
v_start := v_start + 1024;
END LOOP;
utl_file.fclose(v_file);
v_count_object_elab := v_count_object_elab + 1;
END LOOP;
IF v_count_object_elab = 0 THEN
dbms_output.put_line('no mviews');
END IF;
EXCEPTION
WHEN OTHERS THEN
dbms_output.put_line('eror = ' || SQLERRM);
IF utl_file.is_open(v_file) THEN
utl_file.fclose(v_file);
END IF;
RAISE;
END;
/
I need to save a procedure body into a Clob column with a use of variable. String is longer than 4000 characters, so I can't use VarChar2, but with CLOB variable I receive error "ORA-01422: exact fetch returns more than requested number of rows". Same error appears with Varchar2. My PL/SQL block:
DECLARE
txt_procedure CLOB;
BEGIN
SELECT text INTO txt_procedure
FROM all_source
WHERE name = 'My_procedure'
ORDER BY line;
INSERT INTO TABLE1(ID,DATE,CLOB_COLUMN)
VALUES (my_seq.NEXTVAL,'11.10.2018',txt_procedure);
END;
/
How could I insert procedure body into clob column ?
As you will get multiple rows from your query for every line of your source, the following might help:
DECLARE
txt_procedure CLOB;
BEGIN
FOR source_r IN ( SELECT text
FROM all_source
WHERE name = 'My_procedure'
ORDER BY line
)
LOOP
txt_procedure := txt_procedure || chr(10) || source_r.text;
END LOOP;
INSERT INTO TABLE1(ID,DATE,CLOB_COLUMN)
VALUES (my_seq.NEXTVAL,'11.10.2018',txt_procedure);
END;
/
UPDATE
As an alternative, you might also use the DBMS_METADATA package for this:
DECLARE
txt_procedure CLOB;
BEGIN
txt_procedure := DBMS_METADATA.get_ddl(
object_type => 'PROCEDURE',
name => 'My_procedure',
owner => 'YOUR_SCHEMA'
);
INSERT INTO TABLE1(ID,DATE,CLOB_COLUMN)
VALUES (my_seq.NEXTVAL,'11.10.2018',txt_procedure);
END;
/
I am working on something where I have to convert XML type data to Raw but I am not getting it.
I have tried converting the XML data to varchar2 and then using Cast_to_raw but it is showing hex to raw conversion error. Kindly let me know how to do it.
Example describes conversion from xmltype to blob element. If you have to obtain raw value do the dbms_substr.
declare
dest_offset integer;
src_offset integer;
lang_context integer;
warning varchar2(1000);
v_xml_object xmltype;
v_blob blob;
v_clob clob;
begin
--get example xmltype
select dbms_xmlgen.getXmlType('select * from user_objects where rownum < 10') into v_xml_object from dual;
--convert xmltype to clob
v_clob := v_xml_object.getClobVal();
--initi variable
dbms_lob.createtemporary(v_blob, FALSE);
dest_offset := 1;
src_offset := 1;
lang_context := 0;
-- convert to a BLOB here:
dbms_lob.converttoblob( v_blob, v_clob, dbms_lob.getlength( v_clob ), dest_offset, src_offset, 0, lang_context, warning );
dbms_output.put_line(warning);
dbms_output.put_line(dbms_lob.substr(v_blob,length(v_blob)));
end;
I have a procedure where I am checking whether there are new codes. If there are then insert them into a table. And also save the new data into a csv or txt file and email them to me.
I can't get the logic for how to redirect the new data to file or just even put the data as simple text in the email. Thanks
create or replace
PROCEDURE new_codes_test( today_date IN VARCHAR2 DEFAULT NULL, v_proc_return OUT NUMBER) AS
sql_str VARCHAR2(4000);
.....
BEGIN
v_start_time := SYSDATE;
v_proc_return := 0;
....
INSERT INTO NEW_CODES_test
SELECT DISTINCT Sy_ID ,P_CODE, SYSDATE
FROM X.B
WHERE NOT EXISTS (SELECT DISTINCT Sy_ID, P_CODE
FROM X.C
WHERE today = today_date) ;
COMMIT;
--SELECT ___ into ___ from X.B;
sql_str := 'UTL_MAIL.send(sender => ''
,recipients => ''
,cc => ''
,subject => 'New codes'
,MESSAGE => '' )';
--EXECUTE IMMEDIATE sql_str;
p_proc_return := v_proc_return;
EXCEPTIONS
....
END;
To write to a file the UTL_FILE package will come in handy. To write an email you'll need to put the text to be sent into some sort of string before passing it to the MESSAGE argument of UTL_MAIL.SEND. You'll also need to be sure UTL_MAIL is installed and set up on your server; see this FAQ.
So something like the following may be useful:
CREATE OR REPLACE FUNCTION NEW_CODES_TEST(today_date IN VARCHAR2)
RETURN NUMBER
AS
strMessage VARCHAR2(32767);
nRows NUMBER := 0;
fHandle UTL_FILE.FILE_TYPE;
BEGIN
v_start_time := SYSDATE;
fHandle := UTL_FILE.FOPEN(someDirectory, someFilename, 'w');
FOR aRow IN (SELECT DISTINCT SY_ID ,P_CODE
FROM X.B
WHERE NOT EXISTS (SELECT DISTINCT Sy_ID, P_CODE
FROM X.C
WHERE today = today_date)
LOOP
INSERT INTO NEW_CODES_test
VALUES (aRow.SY_ID, aRow.P_CODE, SYSDATE);
UTL_FILE.PUT_LINE(fHandle, aRow.SY_ID || ', ' || aRow.P_CODE);
strMessage := strMessage || 'Added ' || aRow.SY_ID || ', ' ||
aRow.P_CODE || CHR(10);
nRows := nRows + 1;
END LOOP;
COMMIT;
UTL_FILE.FCLOSE(fHandle);
UTL_MAIL.SEND(sender => 'me#mycompany.com',
recipients => 'you#someplaceelse.net',
subject => 'New codes',
message => strMessage);
RETURN 0;
EXCEPTIONS
WHEN OTHERS THEN
DBMS_OUTPUT.PUT_LINE('Error: ' || SQLERRM);
RETURN SQLCODE;
END NEW_CODES_TEST;
Share and enjoy.
I need to retrieve from the database a few large wave files and would like to retrieve divided into wave files smaller (about 5Mb). How can I do? I've seen the procedure dbms_lob.read, but this return maximum file size of 32Kb.
Regards
procedure extract_blob(p_id in number, wrote_length in out number, chunk out blob) is
len_file binary_integer;
myblob blob;
myname varchar2(255);
buffer_length number := 32760;
begin
select file_wav_data, file_wav_name, dbms_lob.getlength(file_wav_data)
into myblob, myname, lun_file
from t_wav
where id = p_id;
if(len_file > wrote_length) then
dbms_lob.read(myblob,buffer_length,wrote_length+1,chunk);
wrote_length := wrote_length + buffer_length;
else wrote_length := -999; --EOF
end if;
end;
You probably want to use temporary LOBs:
procedure extract_blob(
p_id in number,
offset in number,
chunk_length in out number,
chunk out blob
) is
chunk blob;
wav_data blob;
full_length number;
chunk_length number;
begin
select file_wav_data, dbms_lob.getlength(file_wav_data)
into wav_data, full_length
from t_wav
where id = p_id;
chunk_length := greatest(full_length - offset, 0);
if chunk_length = 0 then
return;
end if;
dbms_lob.createtemporary(chunk, TRUE);
dbms_lob.copy(chunk, wav_data, chunk_length, 0, offset);
end extract_blob;
If possible, you should free the temporary LOB from the client side after you have processed it (using DBMS_LOB.FREETEMPORARY).