I remember this schema from QMK: Quantum Mechanical Keyboard where, in order to make the correct keyboard, one wrote something like:
make keyboard some_specs::subtype qualifier
Which has since been replaced by the qmk CLI.
In principle, I agree with writing a CLI for a project as important to keyboard hobbyists as QMK. However, as one person, the CLI becomes a huge Yak Shaving Task (e.g. Like This). And yes, Yak shaving has enormous potential; after all, the html (and javascript) were both yaks (HTML is a shaved Yak, JavaScript -- and later, Rust -- are both Mozilla Yaks).
However, when in pursuit of a different sort of objective, I'd like a good stand in for yak shaving w.r.t. meta-project oriented command line interfaces. My "Go-to" productivity enhancing tool is "GNU-Make" for this particular task.
For example, I have a jupyter notebook directory that I'd like to be cross platform compatible (whether on windows, make or various linux systems), self installing, self configuring, and for it to please hide away corporate jargon (aka "jupyter", "notebook", and "lab"), so as to buffer my precious and limited muscle memory bandwidth from capitalization efforts.
For this reason, I might write -- and have written -- the following make control structure:
make install
make run tgt=foo.ipynb pwd=bar [browser=whatever.exe]
# [...] = optional
My question amounts to: is there a good or make-ish way to write:
make run foo.ipynb bar whatever.exe
Or, as in QMK, is there a good way to go about defining a grammar that make was sort of "designed" to handle?
Related
I'm currently developing a console application with a text-based user interface, in python, which is meant to be an open-source, UNIX alternative for a series of industry standard software and to minimize, in my opinion, the hardware/software requirements for a software of such scale.
However, I do not prefer using curses/ncurses in order to understand the principles of how such libraries work and implement them for my own purposes. For that I've been - let's say - experimenting with ANSI Escape Sequences, for a while.
At some point, I've felt the need that I should split (or to describe in UNIX terminology, multiplex) my terminal into sessions, windows or panes as in GNU Screen or tmux. However, for the reasons I mentioned above, instead of using them in my project I tried to discover their source to get inspired. But the truth is that they both require an extensive knowledge of C, at least for me at this point, and searching through the source codes is exhaustive.
I've recenlty found out that GNU Screen documentation refers to ECMA 48 and ANSI X3.64.
Screen User's Manual
Each virtual terminal provides the functions of the DEC VT100 terminal
and, in addition, several control functions from the ISO 6429 (ECMA
48, ANSI X3.64) and ISO 2022 standards (e.g. insert/delete line and
support for multiple character sets). There is a scrollback history
buffer for each virtual terminal and a copy-and-paste mechanism that
allows the user to move text regions between windows.
I also checked ECMA 48 and ANSI X3.64 for what I want to achieve, couldn't find a clue.
My questions actually arise at this point.
How can I find out which coded character sets in those standards are implemented? For example in §8.3.123 of ECMA 48 refers to DEVICE COMPONENT SELECT MODE, which I could found no signs of implementation or use thereof.
What is the working principle of Screen or tmux in creating a window and pane? What kind of path do they follow for creating windows or panes.
The time I've spent for the second question is quite a lot, and took me nowhere. I thought they may be sweeping the whole screen, drawing borders, delimiting lines or columns with coded characters each time there needs to be an update (resizing, creating a new pane etc.). I considered this option may be a basic but frustrating solution. And I could not be sure that screen or tmux works in that way. But I am pretty sure that I am missing a critical point here.
Any help, opinion or recommendations are appreciated.
I am preparing my opencl accelerated toolkit for release. Currently, I compile my opencl kernels into binaries targeted for a particular card.
Are there other ways of discouraging reverse engineering? Right now, I have many opencl binaries in my release folder, one for each kernel. Would it be better to splice these binaries into one single binary, or even add them into the host binary, and somehow read them in using a special offset ?
OpenCL 2.0 and SPIR-V can be used for this, but is not available on all platforms yet.
Encode binaries. Keep keys in server and have clients request it at time of usage. Ofcourse keys should be encoded too,( using a variable value such as time of server maybe). Then decode in client to use as binary kernel.
I'm not encode pro but I would use multiple algorithms applied multiple times to make it harder, if they are crunchable in several months(needed for new version update of your GPGPU software for example) when they are alone. But simple unknown algorithm of your own such as reversing order of bits of all data (1st bit goes nth position, nth goes 1st) should make it look hard for level-1 hackers.
Warning: some profiling tools could get its codes in "run-time" so you should add many maybe hundreds of trivial kernels without performance penalty to hide it in a crowded timeline or you could disable profiling in kernel options or you could add a deliberate error maybe some broken events in queues then restart so profiler cannot initiate.
Maybe you could obfuscate final C99 code so it becomes unreadable by humans. If can, he/she doesn't need hacking in first place.
Maybe most effectively, don't do anything, just buy copyrights of your genuine algorithm and show it in a txt so they can look but can not dare copying for money.
If kernel can be rewritten into an "interpreted" version without performance penalty, you can get bytecodes from server to client, so when client person checks profiler, he/she sees only interpreter codes but not real working algorithm since it depends on bytecodes from server as being "data"(buffer). For example, c=a+b becomes if(..)else if(...)else(case ...) and has no meaning without a data feed.
On top of all these, you could buy time against some evil people reverseengineer it, you could pick variable names to initiate his/her "selective perception" so he/she can't focus for a while. Then you develop a meaner version at the same time. Such as c=a+b becomes bulletsToDevilsEar=breakDevilsLeg+goodGame
I hear the term "BootStrap" thrown around a lot, but I'm not really sure what it refers to. I know there is a bootstrap CSS, but what exactly does the term mean?
Literally, a bootstrap is a tab on the sides or back of boots that helps you to pull them on. Putting on your shoes or boots is usually the last step of getting dressed; similarly, in programming it's been applied to the initialization or start-up step of a program.
See also the Wikipedia entry for bootstrapping:
Bootstrapping or booting refers to a group of metaphors which refer to a self-sustaining process that proceeds without external help.
[.. in Software Loading] booting is the process of starting a computer, specifically in regards to starting its software. The process involves a chain of stages, in which at each stage a smaller simpler program loads and then executes the larger more complicated program of the next stage. It is in this sense that the computer "pulls itself up by its bootstraps", i.e. it improves itself by its own efforts
[.. in Software Development] bootstrapping can also refer to the development of successively more complex, faster programming environments. The simplest environment will be, perhaps, a very basic text editor (e.g., ed) and an assembler program. Using these tools, one can write a more complex text editor, and a simple compiler for a higher-level language and so on, until one can have a graphical IDE and an extremely high-level programming language.
A shoehorn is another means to help you don footwear but it's idiomatically come to mean cramming something into a tight space.
In computer science Bootstrap (or more commonly "Boot") generally refers to the setup/start/initialization step of a process. It can mean many things depending on the context: starting a physical machine, setting up variables and services for an application to use, or even laying the css groundwork for a website to implement.
Bootstrapping let you create your own most complex design by just minimal configuration, rather than develop it from the scratch.
There are many nice things to like about Makefiles, and many pains in the butt.
In the course of doing various project (I'm a research scientist, "data scientist", or whatever) I often find myself starting out with a few data objects on disk, generating various artifacts from those, generating artifacts from those artifacts, and so on.
It would be nice if I could just say "this object depends on these other objects", and "this object is created in the following manner from these objects", and then ask a Make-like framework to handle the details of actually building them, figuring out which objects need to be updated, farming out work to multiple processors (like Make's -j option), and so on. Makefiles can do all this - but the huge problem is that all the actions have to be written as shell commands. This is not convenient if I'm working in R or Perl or another similar environment. Furthermore, a strong assumption in Make is that all targets are files - there are some exceptions and workarounds, but if my targets are e.g. rows in a database, that would be pretty painful.
To be clear, I'm not after a software-build system. I'm interested in something that (more generally?) deals with dependency webs of artifacts.
Anyone know of a framework for these kinds of dependency webs? Seems like it could be a nice tool for doing data science, & visually showing how results were generated, etc.
One extremely interesting example I saw recently was IncPy, but it looks like it hasn't been touched in quite a while, and it's very closely coupled with Python. It's probably also much more ambitious than I'm hoping for, which is why it has to be so closely coupled with Python.
Sorry for the vague question, let me know if some clarification would be helpful.
A new system called "Drake" was announced today that targets this exact situation: http://blog.factual.com/introducing-drake-a-kind-of-make-for-data . Looks very promising, though I haven't actually tried it yet.
This question is several years old, but I thought adding a link to remake here would be relevant.
From the GitHub repository:
The idea here is to re-imagine a set of ideas from make but built for R. Rather than having a series of calls to different instances of R (as happens if you run make on R scripts), the idea is to define pieces of a pipeline within an R session. Rather than being language agnostic (like make must be), remake is unapologetically R focussed.
It is not on CRAN yet, and I haven't tried it, but it looks very interesting.
I would give Bazel a try for this. It is primarily a software build system, but with its genrule type of artifacts it can perform pretty arbitrary file generation, too.
Bazel is very extendable, using its Python-like Starlark language which should be far easier to use for complicated tasks than make. You can start by writing simple genrule steps by hand, then refactor common patterns into macros, and if things become more complicated even write your own rules. So you should be able to express your individual transformations at a high level that models how you think about them, then turn that representation into lower level constructs using something that feels like a proper programming language.
Where make depends on timestamps, Bazel checks fingerprints. So if at any one step produces the same output even though one of its inputs changed, then subsequent steps won't need to get re-computed again. If some of your data processing steps project or filter data, there might be a high probability of this kind of thing happening.
I see your question is tagged for R, even though it doesn't mention it much. Under the hood, R computations would in Bazel still boil down to R CMD invocations on the shell. But you could have complicated muliti-line commands assembled in complicated ways, to read your inputs, process them and store the outputs. If the cost of initialization of the R binary is a concern, Rserve might help although using it would make the setup depend on a locally accessible Rserve instance I believe. Even with that I see nothing that would avoid the cost of storing the data to file, and loading it back from file. If you want something that avoids that cost by keeping things in memory between steps, then you'd be looking into a very R-specific tool, not a generic tool like you requested.
In terms of “visually showing how results were generated”, bazel query --output graph can be used to generate a graphviz dot file of the dependency graph.
Disclaimer: I'm currently working at Google, which internally uses a variant of Bazel called Blaze. Actually Bazel is the open-source released version of Blaze. I'm very familiar with using Blaze, but not with setting up Bazel from scratch.
Red-R has a concept of data flow programming. I have not tried it yet.
How can I package my Java application into an executable jar that cannot be decompiled (for example , by Jadclipse)?
You can't. If the JRE can run it, an application can de-compile it.
The best you can hope for is to make it very hard to read (replace all symbols with combinations of 'l' and '1' and 'O' and '0', put in lots of useless code and so on). You'd be surprised how unreadable you can make code, even with a relatively dumb translation tool.
This is called obfuscation and, while not perfect, it's sometimes adequate.
Remember, you can't stop the determined hacker any more than the determined burglar. What you're trying to do is make things very hard for the casual attacker. When presented with the symbols O001l1ll10O, O001llll10O, OO01l1ll10O, O0Ol11ll10O and O001l1ll1OO, and code that doesn't seem to do anything useful, most people will just give up.
First you can't avoid people reverse engineering your code. The JVM bytecode has to be plain to be executed and there are several programs to reverse engineer it (same applies to .NET CLR). You can only make it more and more difficult to raise the barrier (i.e. cost) to see and understand your code.
Usual way is to obfuscate the source with some tool. Classes, methods and fields are renamed throughout the codebase, even with invalid identifiers if you choose to, making the code next to impossible to comprehend. I had good results with JODE in the past. After obfuscating use a decompiler to see what your code looks like...
Next to obfuscation you can encrypt your class files (all but a small starter class) with some method and use a custom class loader to decrypt them. Unfortunately the class loader class can't be encrypted itself, so people might figure out the decryption algorithm by reading the decompiled code of your class loader. But the window to attack your code got smaller. Again this does not prevent people from seeing your code, just makes it harder for the casual attacker.
You could also try to convert the Java application to some windows EXE which would hide the clue that it's Java at all (to some degree) or really compile into machine code, depending on your need of JVM features. (I did not try this.)
GCJ is a free tool that can compile to either bytecode or native code. Keeping in mind, that does sort of defeat the purpose of Java.
A little late I know, but the answer is no.
Even if you write in C and compile to native code, there are dissasemblers / debuggers which will allow people to step through your code. Granted - debugging optimized code without symbolic information is a pain - but it can be done, I've had to do it on occasion.
There are steps that you can take to make this harder - e.g. on windows you can call the IsDebuggerPresent API in a loop to see if somebody is debugging your process, and if yes and it is a release build - terminate the process. Of course a sufficiently determined attacker could intercept your call to IsDebuggerPresent and always return false.
There are a whole variety of techniques that have cropped up - people who want to protect something and people who are out to crack it wide open, it is a veritable arms race! Once you go down this path - you will have to constantly keep updating/upgrading your defenses, there is no stopping.
This not my practical solution but , here i think good collection or resource and tutorials for making it happen to highest level of satisfaction.
A suggestion from this website (oracle community)
(clean way), Obfuscate your code, there are many open source and free
obfuscator tools, here is a simple list of them : [Open source
obfuscators list] .
These tools make your code unreadable( though still you can decompile
it) by changing names. this is the most common way to protect your
code.
2.(Not so clean way) If you have a specific target platform (like windows) or you can have different versions for different platforms,
you can write a sophisticated part of your algorithms in a low level
language like C (which is very hard to decompile and understand) and
use it as a native library in you java application. it is not clean,
because many of us use java for it's cross-platform abilities, and
this method fades that ability.
and this one below a step by step follow :
ProtectYourJavaCode
Enjoy!
Keep your solutions added we need this more.