I have a flat file below. I'm having trouble with the schema with this layout. I switched it around to have a header and a detail and created an application with no problems, but the customer won't change the layout. This is probably pretty basic, but I'm a beginner. How do I take certain fields from this layout and create a header and a detail? The last date field needs to be in the header so you can see how it's random.
PO207730CO|1271|customer 1|john doe|1|161075|161075|BROOM FLAGGED LOBBY|2|5.62|24-Feb-2014|
PO207730CO|1271|customer 1|john doe|2|167316|167316|CLEANER DISPATCH SPRAY HOSPITAL DISINFECTANT W/BLEACH|1|59.84|24-Feb-2014|
PO207730CO|1271|customer 1|john doe|3|162175|162175|DUST PAN LOBBY|2|6.26|24-Feb-2014|
PO207730CO|1271|customer 1|john doe|4|163325|163325|MOP WET LARGE GENERAL-PURPOSE BLUE WB/LP|1|18.45|24-Feb-2014|
PO207730CO|1271|customer 1|john doe|5|164715|164715|SOAP PROVON MEDICATED TFX|1|32.79|24-Feb-2014|
PO207730CO|1271|customer 1|john doe|6|166338|166338|TOWEL MULTI-FOLD SCOTT WHITE|5|18.91|24-Feb-2014|
PO207814CO|1264|customer 2|jane doe|1|Cups||Bib 20x35 2 Ply Lab (756220)|1|17.47|24-Feb-2014|
PO207814CO|1264|customer 2|jane doe|2|Cups||Cup 9oz Translucent (098219)|1|24-Feb-2014|
PO207814CO|1264|customer 2|jane doe|3|Cups||Cup Foam 16oz (177190)|2|35.1|24-Feb-2014|
PO207814CO|1264|customer 2|jane doe|4|Cups||Lid 16/20 Whte Tab W/Sslot (194088)|2|16.57|24-Feb-2014|
PO207814CO|1264|customer 2|jane doe|5|Cups||Tissue 2-Ply 100-Sht (343227)|3|16.38|24-Feb-2014|
The basic problem here is the Flat File Disassembler does not support the concept of splitting/debatching based on changing values, PO207730CO -> PO207814CO for example.
So, you'll have to regroup by PO number at some following step.
You have a few options:
Use a Custom XSLT Map to group the lines based on the PO Number,
then split, by using a Receive Pipeline on an Orchestration for example.
https://social.technet.microsoft.com/wiki/contents/articles/17985.xslt-muenchian-grouping-biztalk-complex-transformation.aspx
Use an xPath Debatching Pattern in an Orchestration. http://www.biztalkgurus.com/biztalk_server/biztalk_2004/m/biztalk_2004_samples/32438.aspx
Either way, you would parse the flat file as you are now, row by row.
Thanks for the replies.
I went with an SSIS package to preload a table and build the pipe delimited file. I have Biztalk then picking up that file and debatching it the way I need it.
Related
This is related to this question on accesing the POD, but it goes further than that. You can easily access the Abstract Syntax Tree of a Perl 6 program using:
perl6 --target=ast -e '"Þor is mighty!".say'
This will print the whole Q abstract syntax tree. It's not too clear how to make this from your own program, or I haven't found how to do it. In fact, the CoreHackers::Q module runs that as an external script. But being able to access it from your own program, like
use QAST; # It does not exist
my $this-qast = QAST::Load("some-external-file.p6") # Would want something like this
would be great. I'm pretty sure it should be possible, at the NQP level and probably in a Rakudo-dependent way. Does someone know hot it goes?
Since QAST is not a part of the Perl 6 language specification, but an internal implementation detail of Rakudo, there's no official way to do this. Eventually there will be an AST form that is part of the language specification, but that doesn't yet exist (the 007 project which is working on exploring this area).
It is, however, possible to obtain the QAST tree by using:
use nqp;
my $ast = nqp::getcomp("perl6").eval("say 42", :target<ast>);
say $ast.dump();
I am getting below error when I copy this value in my RIDE resource file in Robot Framework. Don't know what is wrong with this value.
I am able to save other 900 values in same format but not this one.
ERROR:Data Sanity Check Failed.Reset Changes?
Value I am trying to save is as below:
${MISDOB2ATTMM} Dobson to VF Migration (Manual)
Examples of correct values:
${CHILE} Chile
${CI} Cote d Ivoire (Ivory Coast)
${CN} China, Peoples Republic of
${COSTARICA} Costa recei
The error message you got was from the Text Editor (you did not mention this).
This could be because there may be invisible characters in that variable definition. Maybe indentation, or even a TAB symbol.
Also, that format for variable definition is correct for the section,
*** Variables ***
But not correct for keywords or test case sections.
I recommend to always use the '=' symbol for variables assignments outside the Variables section. For example:
${MISDOB2ATTMM}= Set Variable Dobson to VF Migration (Manual)
If I begin with a wholly Japanese sentence and run it through MeCab, I get something like this:
$ echo "吾輩は猫である" | mecab
吾輩 名詞,代名詞,一般,*,*,*,吾輩,ワガハイ,ワガハイ
は 助詞,係助詞,*,*,*,*,は,ハ,ワ
猫 名詞,一般,*,*,*,*,猫,ネコ,ネコ
で 助動詞,*,*,*,特殊・ダ,連用形,だ,デ,デ
ある 助動詞,*,*,*,五段・ラ行アル,基本形,ある,アル,アル
EOS
If I smash together everything I get from the last column, I get "ワガハイワネコデアル", which I can then feed into a speech synthesis program and get output. Said program, however, doesn't handle English words.
I throw English into MeCab, it manages to tokenise it (probably naively at the spaces), but gives no reading:
$ echo "I am a cat" | mecab
I 名詞,固有名詞,組織,*,*,*,*
am 名詞,一般,*,*,*,*,*
a 名詞,一般,*,*,*,*,*
cat 名詞,固有名詞,組織,*,*,*,*
EOS
I want to get readings for these as well, even if they're not perfect, so that I can get something along the lines of "アイアムアキャット".
I have already scoured the web for solutions and whereas I do find a bunch of web sites which have transliteration that appears to be adequate, I can't find any way to do it in my own code. In a couple of cases, I emailed the site authors and got no response yet after waiting for a few weeks. (Just how far behind on their inboxes are these people?)
There are a number of directions I can go but I hit dead ends on all of them so far, so this is my compound question:
MeCab takes custom dictionaries. Is there a custom dictionary which fills in the English knowledge somewhat?
Is there some other library or tool that can take English and spit out Katakana?
Is there some library or tool that can take IPA (International Phonetic Alphabet) and spit out Katakana? (I know how to get from English to IPA.)
As an aside, I find that the software "VOICEROID" can speak English text (poorly, but adequately for my purposes). This software uses MeCab too (or at least its DLL and dictionary files are included in the install.) It also uses another library, Cabocha, which as far as I can tell by running it does the exact same thing as MeCab. It could be using custom dictionaries for either of these two libraries to do the job, or the code to do it could be in the proprietary AITalk library they are using. More research is needed and I haven't figured out how to run either tool against their dictionaries to test it out directly either.
I need to import the Geonames database (http://download.geonames.org/export/dump/) into SQLite (file is about a gigabyte in size, ±8,000,000 records, tab-delimited).
I'm using the built-in SQLite-possibilities of Mac OS X, accessed through terminal. All goes well, until record 381174 (tested with older file, the exact number varies slightly depending on the exact version of the Geonames database, as it is updated every few days), where the error "expected 19 columns of data but found 18" is displayed.
The exact line causing the problem is:
126704 Gora Kyumyurkey Gora Kyumyurkey Gora Kemyurkey,Gora
Kyamyar-Kup,Gora Kyumyurkey,Gora Këmyurkëy,Komur Qu",Komur
Qu',Komurkoy Dagi,Komūr Qū’,Komūr Qū”,Kummer Kid,Kömürköy Dağı,kumwr
qwʾ,كُمور
قوء 38.73335 48.24133 T MT AZ AZ 00 0 2471 Asia/Baku 2014-03-05
I've tested various countries separately, and the western countries all completely imported without a problem, causing me to believe the problem is somewhere in the exotic characters used in some entries. (I've put this line into a separate file and tested with several other database-programs, some did give an error, some imported without a problem).
How do I solve this error, or are there other ways to import the file?
Thanks for your help and let me know if you need more information.
Regarding the question title, a preliminary search resulted in
the GeoNames format description ("tab-delimited text in utf8 encoding")
https://download.geonames.org/export/dump/readme.txt
some libraries (untested):
Perl: https://github.com/mjradwin/geonames-sqlite (+ autocomplete demo JavaScript/PHP)
PHP: https://github.com/robotamer/geonames-to-sqlite
Python: https://github.com/commodo/geonames-dump-to-sqlite
GUI (mentioned by #charlest):
https://github.com/sqlitebrowser/sqlitebrowser/
The SQLite tools have import capability as well:
https://sqlite.org/cli.html#csv_import
It looks like a bi-directional text issue. "كُمور قوء" is expected to be at the end of the comma-separated alternate name list. However, on account of it being dextrosinistral (or RTL), it's displaying on the wrong side of the latitude and longitude values.
I don't have visibility of your import method, but it seems likely to me that that's why it thinks a column is missing.
I found the same problem using the script from the geonames forum here: http://forum.geonames.org/gforum/posts/list/32139.page
Despite adjusting the script to run on Mac OS X (Sierra 10.12.6) I was getting the same errors. But thanks to the script author since it helped me get the sqlite database file created.
After a little while I decided to use the sqlite DB Browser for SQLite (version 3.11.2) rather than continue with the script.
I had errors with this method as well and found that I had to set the "Quote character" setting in the import dialog to the blank state. Once that was done the import from the FULL allCountries.txt file ran to completion taking just under an hour on my MacBookPro (an old one but with SSD).
Although I have not dived in deeper I am assuming that the geonames text files must not be quote parsed in any way. Each line simply needs to be handled as tab delimited UTF-8 strings.
At the time of writing allCountries.txt is 1.5GB with 11,930,517 records. SQLite database file is just short of 3GB.
Hope that helps.
UPDATE 1:
Further investigation has revealed that it is indeed due to the embedded quotes in the geonames files, and looking here: https://sqlite.org/quirks.html#dblquote shows that SQLite has problems with quotes. Hence you need to be able to switch off quote parsing in SQLite.
Despite the 3.11.2 version of DB Browser being based on SQLite 3.27.2 which does not have the required mods to ignore the quotes, I can only assume it must be escaping the quotes when you set the "Quote character" to blank.
I have Vim 7.2 installed on Windows. In GVim, the <C-PageUp> and <C-PageDown> work for navigation between tabs by default. However, it doesn't work for Vim.
I have even added the below lines in _vimrc, but it still does not work.
map <C-PageUp> :tabp<CR>
map <C-PageDown> :tabn<CR>
But, map and works.
map <C-left> :tabp<CR>
map <C-right> :tabn<CR>
Does anybody have a clue why?
The problem you describe is generally caused by vim's terminal settings not knowing the correct character sequence for a given key (on a console, all keystrokes are turned into a sequence of characters). It can also be caused by your console not sending a distinct character sequence for the key you're trying to press.
If it's the former problem, doing something like this can work around it:
:map <CTRL-V><CTRL-PAGEUP> :tabp<CR>
Where <CTRL-V> and <CTRL-PAGEUP> are literally those keys, not "less than, C, T, R, ... etc.".
If it's the latter problem then you need to either adjust the settings of your terminal program or get a different terminal program. (I'm not sure which of these options actually exist on Windows.)
This may seem obvious to many, but konsole users should be aware that some versions bind ctrl-pageup / ctrl-pagedown as secondary bindings to it's own tabbed window feature, (which may not be obvious if you don't use that feature).
Simply clearing them from the 'Configure Shortcuts' menu got them working in vim correctly for me. I guess other terminals may have similar features enabeld by default.
I'm adding this answer, taking details from vi & Vim, to integrate those that are already been given/accepted with some more details that sound very important to me.
The alredy proposed answers
It is true what the other answer says:
map <C-PageUp> :echo "hello"<CR> won't work because Vim doesn't know what escape sequence corresponds to the keycode <C-PageUp>;
one solution is to type the escape sequence explicitly: map ^[[5^ :echo "hello"<CR>, where the escape sequence ^[[5^ (which is in general different from terminal to terminal) can be obtained by Ctrl+VCtrl+PageUp.
One additional important detail
On the other hand the best solution for me is the following
set <F13>=^[[5^
map <F13> :echo "hello"<CR>
which makes use of one of additional function key codes (you can use up to <F37>). Likewise, you could have a bunch of set keycode=escapesequence all together in a single place in your .vimrc (or in another dedicated file that you source from your .vimrc, why not?).