I have a program (not mine - downloaded from i-net) made on ATI streams (more accurate - on brook lang - file is *.br). There is a python script (see below), that compiles it into *.il file using brook compiler, provided by ATI streams SDK. After it script zips it into *.Z file. C-program's Makefile contains this code
my_kernel_dp11.o: my_kernel_dp11.Z
ld -s -r -o my_kernel_dp11.o -b binary my_kernel_dp11.Z
and then it linked to main executed file.
Data from that obj-file read by C-program into some buffer and then called calclCompile function (as I understand it's OpenCL func).
It works fine at AMD HD 6970-series, but failed at AMD HD 7970-series with following error
Unsupported program construct detected in back-end
Here is a python script
#!/usr/bin/python
import sys
import zlib
import os
def makebrz(dp_bits):
try:
os.unlink("a_slice_dpX_a_slicer.il")
except OSError:
pass
dpdefs=""
for i in range(dp_bits-11):
dpdefs = dpdefs + " -D DP_BIT_%i" % (i+12,)
print "DP_DEFS: ", dpdefs
os.system("/usr/local/atibrook/sdk/bin/brcc -k -pp %s a_slice_dpX.br" % (dpdefs,) )
f = open("a_slice_dpX_a_slicer.il")
if f==None:
print "Could not read ", sys.argv[1]
sys.exit(-1)
data = f.read()
f.close()
oname = "../my_kernel_dp%i.Z" % (dp_bits,)
data2 = zlib.compress(data)
fo = open( oname, "wb" )
fo.write(data2)
fo.close()
#os.system("ld -s -r -o ../%s.o -b binary %s" % (oname[:-2],oname))
makebrz(11)
makebrz(12)
makebrz(13)
makebrz(14)
And here is a program http://dl.dropbox.com/u/46469564/a_slice_dpX.br
The question is: what should I do to make it program "supported" ?
P.S. There is one problem - I don't know this technology (brook, ATI streams, OpenCL) at all. That's why advice like "you should try this or that" are useless. I need particular action to do - change this and you'll have a success :)
Thank you.
AFAIK Radeon HD7970 is built on GCN architecture, so if you are using brook to generate IL code, JIT in southern island may not know how to generate proper ISA for the h/w you are using,so if you would like continue using brook+ then you need to wait till an updated version of Brook+ gets released on sourceforge which can generate an IL which gets converted to right ISA(GCN).
Other option is to use AMD APP SDK 2.6 and rewrite your code in OpenCL.
Related
Hello this my first question to StackOverflow, not sure about the forum and topic.
While participating in an Open Mainframe initiative using Visual Studio Code and Putty for Unix I developed a sample program in COBOL showing international sayings (german, english, french, spanish, latin for now). It works fine via batch with JCL to file and being called from REXX. In file I can't see special chars for non-english but I had a lucky punch with a twin-program in PL/1 (doing the same and showing the special chars in REXX).
Now my question: I also tried to call by mvscmd from Unix bash script. It works so far but dont show me the special chars. Ok I have last chance to call mvscmd from Python. Or alternatively I can transfer file from MVS to unix (for any reason then it automatically converts and I see my special chars contents).
Where is the place to handle it? Cobol? (as I said, for any reason PL/1 can do. I only use standard put edit in PL/1 vs display in Cobol). Converting the Sysprint/Sysout?
Any specialist can help me?
Hello and sorry for late replay. Well the whole code is a little bit much but I guess my problem is the following - MVSCMD direct coded in the shell script
#!/bin/sh
parm='Z08800.FYD.DATA'
#echo "arg1=>"$1"<"
[ ! -z "$1" ] && parm=$parm","$1
#echo "arg2=>"$2"<"
[ ! -z "$2" ] && parm=$parm","$2
#echo "parm=>"$parm"<"
mvscmd --pgm=saycob --args=$parm \
--steplib='z08800.fyd.load' \
--sysin=dummy \
--sysout=*
I have some more shell script but this is the main. I directly put it to sysout (its the COBOL diplay. I can use fixed string or my saying read from MVS file). When using PL/1 program the last file is then sysprint because PL/1 makes it by PUT EDIT.
I assume my codepage is pretty wrong. But I dont know how to repair. I used some settings in the shell but LANG remains on C ??? By the way this Unix seems to be quite old and I only have the chance to use it until August.
My main interest is to use the program on Mainframe and in JCL and/or REXX.
But they gave us chance with this embedded Unix (?) also so I wanted to try.
Direct Sysout from COBOL program to Unix terminal.
I meant when executing the program on the Mainframe and then watching the result file in ISPF (old stuff) editor by PF3 I can see German and Spanish and French special characters. So they are there seems, produced by COBOL and PL/1.
When transfering the MVS file (kind of PDS) into the UNIX by MVSCMD, it is also fine (special chars) but thats not what I wanted.
I tried to use Python instead flat shell but its going even worse. I cannot direct the Sysout to terminal, all what is Python able to call is on the Mainframe and with the MVS filesystem. So I have to transfer it after. It is to much overhead in my eyes when call say 7 sayings and I want them to be displayed in the Unix terminal lol.
Here is my REXX that is doing the trick
/* rexx */
ARG PARM1 PARM2
PARAMETER = '/Z08800.FYD.DATA'
If Length(PARM1) > 0
Then PARAMETER = PARAMETER","PARM1
If Length(PARM2) > 0
Then PARAMETER = PARAMETER","PARM2
PARAMETER = "'"PARAMETER"'"
Address TSO "Alloc File(sysprint) Dataset(*)"
Address TSO "Alloc File(sysin) Dummy"
Address TSO "Call fyd.load(saypli)" PARAMETER
Address TSO "Free File(sysprint)"
Address TSO "Free File(sysin)"
It is now the other Load, the PL/1 - but the COBOL does the same with Sysout instead of Sysprint.
It is shown in my REXX terminal that is also called by ISPF and then 3.4 in the edit panel. The program has no manual input but reads file. And yes, the sayings are not allocated here, I read them by dynamic allocation but it doesnt matter from where my strings come to the DISPLAY / PUT EDIT
And this now JCL. OK works little different, it stores to PDS member
//SAYCOB JOB
//COBCLG EXEC IGYWCLG,
// PARM.GO='Z08800.FYD.DATA'
// SET MBR=SAYCOB
//COBOL.SYSIN DD DSN=&SYSUID..FYD.SOURCE(&MBR),DISP=SHR
//LKED.SYSLMOD DD DSN=&SYSUID..FYD.LOAD(&MBR),DISP=SHR
//GO.SYSOUT DD SYSOUT=*
//*-------------------------------------------------------------
//*
//*-------------------------------------------------------------
//SAYCOB EXEC PGM=&MBR,PARM='Z08800.FYD.DATA,001,007'
//STEPLIB DD DSN=&SYSUID..FYD.LOAD,DISP=SHR
//SYSOUT DD DSN=&SYSUID..FYD.OUTPUT(&MBR),DISP=SHR
//*-------------------------------------------------------------
//LIST EXEC PGM=LINE80,PARM='/80'
//STEPLIB DD DSN=&SYSUID..FYD.LOAD,DISP=SHR
//SYSIN DD DSN=&SYSUID..FYD.OUTPUT(&MBR),DISP=SHR
//SYSPRINT DD SYSOUT=*
//
Here in the parameter I give them the library to my sayings and then I allocate by PL/1 or COBOL. I can of course show, but its a little bit much, about 200 lines... The problem is not MVS I guess but the Unix codepage.
I have configured theano as follows:
[idf#localhost python]$ more ~idf/.theanorc
[global]
device = opencl0:0
floatX = float32
[lib]
cnmem=100
[idf#localhost python]$
I also needed to
[idf#localhost python]$ export MKL_THREADING_LAYER=GNU
although interestingly enough, if I install openblas and add
[blas]
ldflags = -lopenblas
to the .theanorc file, I no longer need to:
export MKL_THREADING_LAYER=GNU
Using a program I found on the internet which I modified slightly to use gpuarray, I am attempting to use theano with an Intel GPU through opencl:
import os
import shutil
from theano import function, config, shared, gpuarray
import theano.tensor as T
import numpy
import time
vlen = 10 * 30 * 768 # 10 x #cores x # threads per core
iters = 1000
rng = numpy.random.RandomState(22)
x = shared(numpy.asarray(rng.rand(vlen), config.floatX))
f = function([], T.exp(x))
print(f.maker.fgraph.toposort())
t0 = time.time()
for i in xrange(iters):
r = f()
t1 = time.time()
print("Looping %d times took %f seconds" % (iters, t1 - t0))
print("Result is %s" % (r))
if numpy.any([isinstance(x.op, T.Elemwise) for x in f.maker.fgraph.toposort()]):
print('Used the cpu')
else:
print('Used the gpu')
When I run the program, it seems as if it recognizes the GPU, but at the end the message "used the cpu" is printed.
[idf#localhost python]$ python theanoexam1.py
Mapped name None to device opencl0:0: Intel(R) HD Graphics 5500 BroadWell U-Processor GT2
[GpuElemwise{exp,no_inplace}(<GpuArrayType<None>(float32, vector)>), HostFromGpu(gpuarray)(GpuElemwise{exp,no_inplace}.0)]
Looping 1000 times took 1.231896 seconds
Result is [ 1.23178029 1.61879337 1.52278054 ..., 2.20771813 2.29967737
1.62323284]
Used the cpu
[idf#localhost python]$
I am skeptical of the message "used the cpu": 1.231896 seconds seems fast for an Intel i3 with four cores.
Is there an extra configuration that is needed to use opencl with theano? Or did this program indeed show that theano is configured to use the GPU through opencl?
Firstly thank you for you post.
I am running on Ubuntu 16.04, with Conda, and I have manually installed libgpuarray - all of which is well documented on the web.
I used the same test program you did (thank you for providing it).
So here are my settings
export MKL_THREADING_LAYER=GNU
The file ~/.theanorc looks like this
[global]
device = opencl0:0
floatX = float32
[lib]
cnmem=100
When I run the code
python test.py
I see the output
DRM_IOCTL_I915_GEM_APERTURE failed: Invalid argument
Assuming 131072kB available aperture size.
May lead to reduced performance or incorrect rendering.
get chip id failed: -1 [2]
param: 4, val: 0
Mapped name None to device opencl0:0: Ellesmere
[GpuElemwise{exp,no_inplace}(<GpuArrayType<None>(float32, vector)>), HostFromGpu(gpuarray)(GpuElemwise{exp,no_inplace}.0)]
Looping 1000 times took 0.282664 seconds
Result is [1.2317803 1.6187935 1.5227805 ... 2.207718 2.2996776 1.6232328]
Used the gpu
I can not figure out how to use the 2nd GPU (also OpenCL) - but I am happy at the moment that at least I have 1 GPU running.
I didn't realize that statistical charts and graphs made by R were using fonts not installed on my Windows machine, these are:
Helvetica
Helvetica-Bold
Helvetica-Oblique
ZapfDingbats
I discovered that by running pdffonts <file.name.pdf> from command line. So these fonts were not embedded in the PDF files.
I have my dissertation sent for printing but the printing house says these fonts need to be embedded in the PDF file. I have written it using LaTeX and included graphs as PDFs.
How to replace or substitute these fonts (some are licensed) with very similar ones without distorting the graphs in the individual PDFs? I don't intend to use commercial utilities.
Notes
- Windows 7 32 bit
- This post was similar:
https://superuser.com/a/223687/212779
However, it was done with a commercial program and is relatively old by now. There might be better new ideas to overcome this problem. I hope so.
My trial so far
I have succeeded to do the replacement using the procedure in the following post by some expert in Ghostscript: https://superuser.com/q/39167/212779
This requires Ghostscript to be installed (I have version 9.15) plus Adobe PostScript for Windows, a universal one and this script from command line:
gswin32c ^
-dNOPAUSE ^
-dBATCH ^
-sDEVICE=pdfwrite ^
-dPDFSETTINGS=/prepress ^
-dCompatibilityLevel=1.4 ^
-dHaveTrueTypes=true ^
-dSubsetFonts=true ^
-sOutputFile="c:\path\to\somename.pdf" ^
-c ".setpdfwrite <</NeverEmbed [ ]>> setdistillerparams" ^
-f "c:\path\to\somename.ps"
Question
How to get the generated .ps postscript file and subsequently the .pdf file (which is now gracefully embedded with a similar font) having the same size of the original PDF, i.e., cropped exactly to the same dimensions of the original PDF file?
If I leave all default settings in Adobe Reader (version XI) -- not to be confused with the commercial Adobe Professional, I get the same size in .ps file, so what I really need is some code in Ghostscript to preserved dimensions of the .ps file when generate the final PDF, any help? BTW, I open the .ps file with SumatraPDF viewer.
You were close: a previous SO question (12857849) mentioned adding -dEmbedAllFonts=true to the command line of gswin32c.
Your second question (in a comment) for batch-processing multiple PDFs can be done in multiple ways, the "best" being what is easiest for you to understand, maintain, and using a tool to which you have ready access.
In R:
fnames <- list.files(pattern = 'doctorate-.*.pdf')
for (fn in fnames) {
ofn <- paste0(gsub('\\.pdf$', '', fn, ignore.case = TRUE), '-withfonts.pdf')
message('Processing: ', fn, ' --> ', ofn)
system2('gswin32c',
paste0('-dNOPAUSE -dBATCH -sDEVICE=pdfwrite -dPDFSETTINGS=/prepress ',
'-dCompatibilityLevel=1.4 -dHaveTrueTypes=true -dSubsetFonts=true ',
'-dEmbedAllFonts=true -sOutputFile="', ofn, '" ',
'-c ".setpdfwrite <</NeverEmbed [ ]>> setdistillerparams" ',
'-f "', fn, '"'))
}
This is a bit verbose. If you add stdout=FALSE in system2, all you should see is something like:
Processing: doctorate-1.pdf --> doctorate-1-withfonts.pdf
Processing: doctorate-2.pdf --> doctorate-2-withfonts.pdf
Processing: doctorate-3.pdf --> doctorate-3-withfonts.pdf
In bash:
for fn in doctorate-*.pdf ; do
ofn="$(basename ${fn} .pdf)-withfonts.pdf"
echo "Processing: ${fn} --> ${ofn}"
gswin32c -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -dPDFSETTINGS=/prepress ^
-dCompatibilityLevel=1.4 -dHaveTrueTypes=true -dSubsetFonts=true ^
-dEmbedAllFonts=true -sOutputFile="${ofn}" ^
-c ".setpdfwrite <</NeverEmbed [ ]>> setdistillerparams" ^
-f "${fn}"
done
(Note that if you are doing this in a bash terminal under msys2 in windows, this will likely fail since msys2 sees "/prepress" and assumes it is a path, converting it to c:/msys64/prepress, which obviously means nothing to gs ...)
All what is needed in Windows is the following steps:
Step 1
Install Ghostscript and add to the PATH variables (Win + break is useful shortcut)
Step 2
Run this script from command line and go the target folder where you have the bad PDF (I do this to avoid writing the whole path).
Step 3
Check your bad PDF by running pdffont from command line:
pdffonts input-pdf-where-some-fonts-are-not-embedded.pdf
Step 4
Run this code to get the desired PDF with embedded fonts:
gswin32c ^
-sFONTPATH=C\Windows\Fonts ^
-o output-pdf-with-embedded-fonts.pdf ^
-sDEVICE=pdfwrite ^
-dPDFSETTINGS=/prepress ^
"input-pdf-where-some-fonts-are-not-embedded.pdf"
Is there a way to set the compiler warnings to be interpreted as an error in the Arduino IDE?
Or any generic way to set GCC compiler options?
I have looked at the ~/.arduino/preferences.txt file, but I found nothing that indicates fine-tuned control. I also looked if I could set GCC options via environment variables, but I did not find anything.
I don't want to have verbose compiler output (which you can specify using the IDE) that is way too much distracting non-essential information, and I don't want to waste my time on reading through it.
I want for a compilation to stop on a warning, so code can be cleaned up. My preference would be to be able to set -Werror= options, but a generic -Werror will do for the small code size of .ino projects.
Addendum:
Based on the suggestion in the selected answer, I implemented an avr-g++ script and put that in the path before the normal avr-g++. For that I changed the Arduino command as follows:
-export PATH="${APPDIR}/java/bin:${PATH}"
+export ORGPATH="${APPDIR}/java/bin:${PATH}"
+export PATH="${APPDIR}/extra:${ORGPATH}"
And in the new directory extra in the APPSDIR (where the Arduino command is located), I have
an avr-g++ which is a Python script:
#!/usr/bin/env python
import os
import sys
import subprocess
werr = '-Werror'
wall = '-Wall'
cmd = ['avr-g++'] + sys.argv[1:]
os.environ['PATH'] = os.environ['ORGPATH']
fname = sys.argv[-2][:]
if cmd[-2].startswith('/tmp'):
#print fname, list(fname) # this looks strange
for i, c in enumerate(cmd):
if c == '-w':
cmd[i] = wall
break
cmd.insert(1, werr)
subprocess.call(cmd)
So you replace the first command with the original compiler name and reset the environment used to exclude the extra directory.
The fname is actually strange. If you print it, it is only abc.cpp, but its length is much larger and it actually starts with /tmp. So I check for that to decide whether to add/update the compile options.
It looks like you are on Linux. Arduino is a script, so you can set PATH in the script to include a directory at the beginning to a directory containing a program, avr-g++. Then the Java stuff should take the compiler from there, should it not?
That program then calls the normal /usr/bin/avr-g++ with the extra options.
One option you have is to compile your sketches from the command line. Take a look at this makefile.
I've found this dictionary by William Whitaker on the Internet and I like its parsing capabilities. But the output doesn't fit for me.
The issue (challenge for me):
Given an input form such as "audiam", the program returns the following output (plain text):
audi.am V 4 1 PRES ACTIVE SUB 1 S
audi.am V 4 1 FUT ACTIVE IND 1 S
audio, audire, audivi, auditus V (4th) [XXXAO]
hear, listen, accept, agree with; obey; harken, pay attention; be able to hear;
But I just want to receive the following text output (same input: audiam):
audiam=audio, audire, audivi, auditus
That is:
InputWord=Dictionary_Forms
So some pieces of information are needless for me.
How can I change the output of this program by modifying the Ada code?
I don't have any Ada knowledge, but I know Delphi/Pascal so it's quite easy to understand the code, isn't it? So the parts causing the text output seem to be the TEXT_IO.PUT(...) statements, right? They're all called in list_package.adb so this is probably the source file to look at.
What has to be changed in particular?
The full Ada 95 source code of this program is available on this page.
I hope some of you are able to understand Ada 95 code. Thank you very much in advance!
My compiling problems:
For use on a windows machine, I downloaded MinGW and tried to compile the source files using "MinGW Shell". But this was my input and the shell's reponse:
Compiling with the latest Cygwin version:
When I compile the program using the latest version of Cygwin, there is no error message:
There is even an .exe file which is created. Its size is 1.6 MB (1,682,616 bytes). But when I open it, it closes right away. What has gone wrong?
William Whitaker's Words is a handy tool. You may be able to find a version already built for your platform. I've not changed the code, but you can alter some things using various parameters. It's even hosted online. If you get an Ada compiler, I've included the last Makefile I used. It's a little thin on abstraction, but it includes the essential steps to compile the program and utilities, along with the steps to build the dictionaries.
TARG = words
ARGS = -O
$(TARG): *.ad[bs]
gnatmake $(TARG) $(ARGS)
all: $(TARG)
gnatmake makedict $(ARGS)
gnatmake makeinfl $(ARGS)
gnatmake makestem $(ARGS)
gnatmake makeefil $(ARGS)
#echo Please make the dicitionary
#echo ./makedict DICTLINE.GEN
#echo ./makestem STEMLIST.GEN
#echo ./makeefil EWDSLIST.GEN
#echo ./makeinfl INFLECTS.GEN
debug:
gnatmake -g $(TARG)
clean:
rm -f *.o *.ali b~* core
cleaner: clean
rm -f *.s makedict makeinfl makestem makeefil
cleanest: cleaner
rm -f $(TARG)
Addendum: One approach is to use gcc 4.4.3 on Ubuntu 10.04 with the updated Makefile above. For convenience, I used VirtualBox to host the linux instance.
$ gcc --version
gcc (Ubuntu 4.4.3-4ubuntu5) 4.4.3
Copyright (C) 2009 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Here's a quick test using the title of my second favorite passage from Catulli Carmina.
$ ./words odi et amo
odi V 6 1 PRES ACTIVE IMP 2 S
odeo, odire, odivi(ii), - V TRANS [EXXCW] Later
od.i V 4 1 PRES ACTIVE IMP 2 S
odio, odire, odivi, - V (4th) TRANS [FXXCF] Medieval
hate; dislike; be disinclined/reluctant/adverse to; (usu. PREFDEF);
odi N 2 4 GEN S N Early
odium, odi(i) N (2nd) N [XXXAO]
hate/hatred/dislike/antipathy; odium, unpopularity; boredom/impatience;
hatred (manifested by/towards group), hostility; object of hate/odium;
od.i V 3 1 PERF ACTIVE IND 1 S
odi, odisse, osus V (3rd) PERFDEF [XXXBX]
hate (PERF form, PRES force), dislike; be disinclined/reluctant/adverse to;
et CONJ
et CONJ [XXXAX]
and, and even; also, even; (et ... et = both ... and);
am.o V 1 1 PRES ACTIVE IND 1 S
amo, amare, amavi, amatus V (1st) [XXXAO]
love, like; fall in love with; be fond of; have a tendency to;
Addendum: Once you've got it running, the problem of modifying it remains. A grep for Put_Line\( shows 629 hits; most are in line_stuff and list*. That's where I'd start. As you are learning Ada, there are several good tutorials here.
As much as I like Ada and would encourage you to learn the minimal amount it would require to hack it the way you want...
Really, you are asking for a simple data filter, which it would be quite easy to accomplish by piping your output to awk. If you are running on any flavor of Linux you have awk already (and really should learn to use it). If you are on Windows, you can get awk and all sorts of other useful goodies from MinGW, which is one of the places you'd need to go to get an Ada compiler anyway.
If you do want a Windows Ada compiler, I'd suggest getting GNAT/GCC from there. The two other flavors available, GNAT/GPL and GNAT/PRO are available from AdaCore (the maintainers). However, GNAT/PRO must be purchased and GNAT/GPL renders distributions of any program compiled using it GPL. You might not mind the GPL applying to your program I suppose, but I'm guessing this isn't a serious enough need to spring for commercial support.
If you are on Linux, the GNAT Ada compiler should be available with GCC as an option (if not installed by default). The same two other options from AdaCore are available there too of course, if you like.
Well, you asked about learning Ada. Really, if you are familiar with other compiled procedural languages (eg: C/C++, Java, Pascal, Modula-2, etc.) you shouldn't have too much trouble picking it up. This question covers Ada books. For myself, I generally just use the official LRM as a reference. Unlike most languages, Ada has an internationally standardized Language Reference Manual that is available online for free. It is also quite readable, as such things go.
About compiling: you can use GNAT. It supports Ada83, Ada95, and Ada05. To tell it to use Ada95, use the -gnat95 switch.
You can get it on http://libre.adacore.com/