libxml2 XML_PARSE_HUGE option for xmlParseMemory - libxml2

C++ on Centos 6.4, libxml2.x86_64 2.7.6-12.el6_4.1:
I'm trying to fix an old C++ program that occasionally gets XML parser errors on large xml files, seems to need the XML_PARSE_HUGE option set. But I can't see any place to set it! The code that's failing is using the xmlParseMemory function which only has 2 parameters - the char array to parse and its size.
Is there some way to set the XML_PARSE_HUGE option globally?

You have to switch to xmlReadMemory which has an options parameter. Simply convert calls like
xmlParseMemory(buffer, size);
to
xmlReadMemory(buffer, size, NULL, NULL, XML_PARSE_HUGE);
(I think xmlParseMemory predates the parser options and is only retained for backward compatibility. Also see this question.)

Related

Increase maxTokensPerLine in Atom.IO

How can I increase maxTokensPerLine in my own Atom.IO environment?
I've got some long lines causing syntax to not be recognized properly, for example not highlighted correctly and brackets not taken note of etc.
But this seems to be a current source containing it. It seems to be taken as a parameter which suggests it could be configurable?
grammar-registry.coffee
I found
this.maxTokensPerLine = (_ref1 = options.maxTokensPerLine) != null ? _ref1 : Infinity;
on line 22 of /usr/share/atom/resources/app/apm/node_modules/first-mate/lib/grammar-registry.js
maxTokensPerLine also appears in
/usr/share/atom/resources/app/apm/node_modules/first-mate/lib/grammar.js
I tried adding maxTokensPerLine: 1000 in config.cson under *, core and editor, but it had no effect.
(old) maxTokensPerLine
syntax.coffee
You can use the package grammar-token-limit, which will handle changing it for you. All you need to do is specify which value you want in the package settings.
I guess if you want to do it yourself, this package would be the place to start looking.

Qt error is printed on the console; how to see where it originates from?

I'm getting this on the console in a QML app:
QFont::setPointSizeF: Point size <= 0 (0.000000), must be greater than 0
The app is not crashing so I can't use the debugger to get a backtrace for the exception. How do I see where the error originates from?
If you know the function the warning occurs in (in this case, QFont::setPointSizeF()), you can put a breakpoint there. Following the stack trace will lead you to the code that calls that function.
If the warning doesn't include the name of the function and you have the source code available, use git grep with part of the warning to get an idea of where it comes from. This approach can be a bit of trial and error, as the code may span more than one line, etc, and so you might have to try different parts of the string.
If the warning doesn't include the name of the function, you don't have the source code available and/or you don't like the previous approach, use the QT_MESSAGE_PATTERN environment variable:
QT_MESSAGE_PATTERN="%{function}: %{message}"
For the full list of variables at your disposal, see the qSetMessagePattern() docs:
%{appname} - QCoreApplication::applicationName()
%{category} - Logging category
%{file} - Path to source file
%{function} - Function
%{line} - Line in source file
%{message} - The actual message
%{pid} - QCoreApplication::applicationPid()
%{threadid} - The system-wide ID of current thread (if it can be obtained)
%{qthreadptr} - A pointer to the current QThread (result of QThread::currentThread())
%{type} - "debug", "warning", "critical" or "fatal"
%{time process} - time of the message, in seconds since the process started (the token "process" is literal)
%{time boot} - the time of the message, in seconds since the system boot if that can be determined (the token "boot" is literal). If the time since boot could not be obtained, the output is indeterminate (see QElapsedTimer::msecsSinceReference()).
%{time [format]} - system time when the message occurred, formatted by passing the format to QDateTime::toString(). If the format is not specified, the format of Qt::ISODate is used.
%{backtrace [depth=N] [separator="..."]} - A backtrace with the number of frames specified by the optional depth parameter (defaults to 5), and separated by the optional separator parameter (defaults to "|"). This expansion is available only on some platforms (currently only platfoms using glibc). Names are only known for exported functions. If you want to see the name of every function in your application, use QMAKE_LFLAGS += -rdynamic. When reading backtraces, take into account that frames might be missing due to inlining or tail call optimization.
On an unrelated note, the %{time [format]} placeholder is quite useful to quickly "profile" code by qDebug()ing before and after it.
I think you can use qInstallMessageHandler (Qt5) or qInstallMsgHandler (Qt4) to specify a callback which will intercept all qDebug() / qInfo() / etc. messages (example code is in the link). Then you can just add a breakpoint in this callback function and get a nice callstack.
Aside from the obvious, searching your code for calls to setPointSize[F], you can try the following depending on your environment (which you didn't disclose):
If you have the debugging symbols of the Qt libs installed and are using a decent debugger, you can set a conditional breakpoint on the first line in QFont::setPointSizeF() with the condition set to pointSize <= 0. Even if conditional breakpoints don't work you should still be able to set one and step through every call until you've found the culprit.
On Linux there's the tool ltrace which displays all calls of a binary into shared libs, and I suppose there's something similar in the M$ VS toolbox. You can grep the output for calls to setPointSize directly, but of course this won't work for calls within the lib itself (which I guess could be the case when it handles the QML internally).

OpenCL compiler white-space problems

I'm trying to get started with OpenCL but came across weird behavior of the OpenCL compiler with respect to white-space and can't seem to find any documentation about that.
C-style single-line comments (// foo) immediately cause a meaningless build error: At end of source: error: expected a "}". Multi-line comments (/* bar */) seem to work fine.
Line breaks seem to get stripped without adding whitespace which can cause errors. This example will not compile because of that:
__kernel
void TestKernel() {}
line 1: error: identifier "__kernelvoid" is undefined
This may totally depend on my machine and/or configuration but can somebody confirm that these things should not be this way?
I am using OpenCL via Cloo from .net/C#. The driver is from AMD OpenCL 2.0 AMD-APP (1642.5)
I think I figured it out. I was doing this:
var program = new ComputeProgram(context, File.ReadAllLines(filename));
File.ReadAllLines() returns an array of strings without the line-break characters which is the root of the errors I was getting.
Using File.ReadAllTest() instead fixed all the problems:
var program = new ComputeProgram(context, File.ReadAllText(filename));
But in my opinion some of the blame goes to either Cloo or the OpenCL API for accepting a string array but just concatenating it together..

Unix write() function (libc)

I am making a C application in Unix that uses raw tty input.
I am calling write() to characters on the display, but I want to manipulate the cursor:
ssize_t
write(int d, const void *buf, size_t nbytes);
I've noticed that if buf has the value 8 (I mean char tmp = 8, then passing &tmp), it will move the cursor/pointer backward on the screen.
I was wondering where I could find all the codes, for example, I wish to move the cursor forward but I cannot seem to find it via Google.
Is there a page that lists all the code for the write() function please?
Thank you very much,
Jary
8 is just the ascii code for backspace. You can type man ascii and look at all the values (the man page on my Ubuntu box has friendlier names for the values). If you want to do more complicated things you may want to look at a library like ncurses.
You have just discovered that character code 8 is backspace (control-H).
You would probably be best off using the curses library to manage the screen. However, you can find out what control sequences curses knows about by using infocmp to decompile the terminfo entry for your terminal. The format isn't particularly easy to understand, but it is relatively comprehensive. The alternative is to find a manual for the terminal, which tends to be rather hard.
For instance, I'm using a color Xterm window; infocmp says:
# Reconstructed via infocmp from file: /usr/share/terminfo/78/xterm-color
xterm-color|nxterm|generic color xterm,
am, km, mir, msgr, xenl,
colors#8, cols#80, it#8, lines#24, ncv#, pairs#64,
acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
bel=^G, bold=\E[1m, clear=\E[H\E[2J, cr=^M,
csr=\E[%i%p1%d;%p2%dr, cub=\E[%p1%dD, cub1=^H,
cud=\E[%p1%dB, cud1=^J, cuf=\E[%p1%dC, cuf1=\E[C,
cup=\E[%i%p1%d;%p2%dH, cuu=\E[%p1%dA, cuu1=\E[A,
dch=\E[%p1%dP, dch1=\E[P, dl=\E[%p1%dM, dl1=\E[M, ed=\E[J,
el=\E[K, enacs=\E)0, home=\E[H, ht=^I, hts=\EH, il=\E[%p1%dL,
il1=\E[L, ind=^J,
is2=\E[m\E[?7h\E[4l\E>\E7\E[r\E[?1;3;4;6l\E8, kbs=^H,
kcub1=\EOD, kcud1=\EOB, kcuf1=\EOC, kcuu1=\EOA,
kdch1=\E[3~, kf1=\E[11~, kf10=\E[21~, kf11=\E[23~,
kf12=\E[24~, kf13=\E[25~, kf14=\E[26~, kf15=\E[28~,
kf16=\E[29~, kf17=\E[31~, kf18=\E[32~, kf19=\E[33~,
kf2=\E[12~, kf20=\E[34~, kf3=\E[13~, kf4=\E[14~,
kf5=\E[15~, kf6=\E[17~, kf7=\E[18~, kf8=\E[19~, kf9=\E[20~,
kfnd=\E[1~, kich1=\E[2~, kmous=\E[M, knp=\E[6~, kpp=\E[5~,
kslt=\E[4~, meml=\El, memu=\Em, op=\E[m, rc=\E8, rev=\E[7m,
ri=\EM, rmacs=^O, rmcup=\E[2J\E[?47l\E8, rmir=\E[4l,
rmkx=\E[?1l\E>, rmso=\E[m, rmul=\E[m,
rs2=\E[m\E[?7h\E[4l\E>\E7\E[r\E[?1;3;4;6l\E8, sc=\E7,
setab=\E[4%p1%dm, setaf=\E[3%p1%dm, sgr0=\E[m, smacs=^N,
smcup=\E7\E[?47h, smir=\E[4h, smkx=\E[?1h\E=, smso=\E[7m,
smul=\E[4m, tbc=\E[3g, u6=\E[%i%d;%dR, u7=\E[6n,
u8=\E[?1;2c, u9=\E[c,
That contains information about box drawing characters, code sequences generated by function keys, various cursor movement sequences, and so on.
You can find out more about X/Open Curses (v4.2) in HTML. However, that is officially obsolete, superseded by X/Open Curses v7, which you can download for free in PDF.
If you're using write just so you have low-level cursor control, I think you are using the wrong tool for the job. There are command codes for many types of terminal. VT100 codes, for example, are sequences of the form "\x1b[...", but rather than sending raw codes, you'd be much better off using a library like ncurses.

console print w/o scrolling

I see console apps print colors and seen apps such as ffmpeg print text over itself instead of a new line. How do I print over an existing line? I want to display fps in my console app either at the very top or very bottom and have regular printfs go there and scroll normally.
I need this for windows, but this is meant to be cross platform, so I will eventually have a linux and mac implementation.
There is two simple possibilities which work on linux as well as windows, but only for one line:
printf("\b"); will return for one character, so you might count how many character you want to backspace and fire this in a loop, or you know that you only write n numbers and do it likeprintf("\b\b\b\b\b\b\b\b\b\b");
printf("text to be overwritten by next printf\r"); this will return the cursor to the beginning of the line, so any next printf will overwrite it. Make sure to write a string of same length or longer so you overwrite it entirely.
If you want to rewrite several lines, there is nothing so portable as ncurses, there is libs for it on practically every operating system, and you don't have to take care of the ANSI-differences.
edit: added link to ncurses wikipedia page, gives great overview and introduction, as well as link list and maybe a translation to your preferred language
Check out ncurses. It has bindings for most scripting languages.
You can use '\r' instead of '\n'.
The ASCII character number 8 (A.K.A. Ctrl-H, BS or Backspace) lets you back up one character. ASCII Character number 13 (A.K.A Ctrl-M, CR or Carriage Return) returns the cursor at the beggining of the line.
If you are working in C try putchar(8); and putchar(13);
The magic of the colors, cursor locating and bliking and so on are inside ANSI escape codes. Any text console capable of handling ANSI codes can use them just printing them out to console (i.e. by means of echo in a bash script or printf() function in C).
Unix terminals support ANSI escape sequences and Windows world used to support them back in old MS-DOS days, but the multibyte console support put an end to this. There is more information here. However there are other ways out of just ANSI sequences printing available on Windows. Moreover if you have Cygwin installed on your Windows maching ANSI codes work just as great as on any Unix terminal.
Many people mention Ncurses library that is the de-facto standard for any gui-like text based applications. What this library does is to hide all the terminal differences (Windows/Unix flavours) to represent the same information as identical as possible across all the platforms, though from my own experience I tell you this is not always true (i.e. typical text window frames change because the especial chars are not available under all character encodings). The counterpart of using ncurses is that it is a complete API and it is much harder to start out with it than simply writing out some ANSI escape sequences for simple things such as change the font color, cleaning screen or moving back the cursor to a random position.
For the sake of completeness I paste an example of use of ANSI sequence under Linux that changes the prompt to blue and shows the date:
PS1="\[\033[34m\][\$(date +%H%M)][\u#\h:\w]$ "
You can use Ncurses -
ncurses package is a subroutine library for terminal-independent screen-painting and input-event handling which presents a high level screen model to the programmer, hiding differences between terminal types and doing automatic optimization of output to change one screenfull of text into another
Depending on the platform which you are developing on there's probably a more powerful API which you could use, rather than old ASCII control codes.
e.g. If you are working on Win32 you can actually manipulate the console screen buffer directly.
A good place to start might be here
http://msdn.microsoft.com/en-us/library/ms683171(VS.85).aspx
I have been looking for similar functions/API which would allow me to access the console as something other than a stream of text for other platforms. Haven't found anything yet, but then again, I haven't been looking that hard.
Hope it helps.

Resources