I need to check to see if a file is a valid pgp encrypted file or not. Some pgp files that we get have an extension of pgp and some dont. I need to check to see which of the files are pgp encrypted files and which are not. Please let me know if there is a way to tell.
The only certain way is to attempt decrypting the file (e.g. with gpg) and interpret the error output. Unless you limit input to ascii-armored files, in that case you can check for the armor.
The python-gpgme library is a Pythonic wrapper for GPGME, the library allowing programmatic GnuPG access.
If you have some files that may or may not be GnuPG encrypted:
$ head --bytes=1024k < /dev/urandom > lorem
$ head --bytes=1024k < /dev/urandom | gpg --encrypt --recipient DEADBEEF > ipsum
With the gpgme module you can attempt to decrypt the files:
import gpgme
import io
context = gpgme.Context()
for infile_path in ['lorem', 'ipsum']:
with open(infile_path, 'rb') as infile:
outfile = io.BytesIO()
try:
context.decrypt(infile, outfile)
except gpgme.GpgmeError as exc:
if exc.code == gpgme.ERR_NO_DATA:
print(
"Not a GnuPG-encrypted file: ‘{path}’ ({error})".format(
path=infile.name, error=exc.strerror))
else:
print(
"Error decrypting file: ‘{path}’ ({error})".format(
path=infile.name, error=exc.strerror))
else:
print("Successfully decrypted: ‘{path}’".format(
path=infile.name))
That lets you handle three conditions:
The gpgme.Context.decrypt method fails, and the error code is gpgme.ERR_NO_DATA. This means the data stream was not recognised as GnuPG-encrypted data.
The gpgme.Context.decrypt method fails for some other reason. You'll need to decide which other errors you care about here.
The gpgme.Context.decrypt method succeeds. Obviously, the file is a correctly-encrypted file.
Related
So I am looking at my professor's code that he handed out to try and give us an idea of how to implement >, <, | support into our unix shell. I ran his code and was amazed at what actually happened.
if( pid == 0 )
{
close(1); // close
fd = creat( "userlist", 0644 ); // then open
execlp( "who", "who", NULL ); // and run
perror( "execlp" );
exit(1);
}
This created a userlist file in the directory I was currently in, with the "who" data inside that file. I don't see where any connection between fd, and execlp are being made. How did execlp manage to put the information into userlist? How did execlp even know userlist existed?
Read Advanced Linux Programming. It has several chapters related to the issue. And we cannot explain all this in a few sentences. See also the standard stream and process wikipages.
First, all the system calls (see syscalls(2) for a list, and read the documentation of every individual system call that you are using) your program is doing should be tested against failure. But assume they all succeed. After close(1); the file descriptor 1 (STDOUT_FILENO) is free. So creat("userlist",0644) is likely to re-use it, hence fd is 1; you have redirected your stdout to the newline created userlist file.
At last, you are calling execlp(3) which will call execve(2). When successful, your entire process is restarted with the new executable (so a fresh virtual address space is given to it), and its stdout is still the userlist file descriptor. In particular (unless execve fails) the perror call is not reached.
So your code is a bit what a shell running who > userlist is doing; it does a redirection of stdout to userlist and runs the who command.
If you are coding a shell, use strace(1) -notably with -f option- to understand what system calls are done. Try also strace -f /bin/sh -c ls to look into the behavior of a shell. Study also the source code of existing free software shells (e.g. bash and sash).
See also this and the references I gave there.
execlp knowns nothing. Before execing stdout was closed and a file opened, so the descriptor is the one corresponding to stdout (opens always returns the lowest free descriptor). At that point the process has an "stdout" plugged to the file. Then exec is called and this replaces to whole address space, but some properties remains as the descriptors, so know the code of who is executed with an stdout that correspond to the file. This is the way redirections are managed by shells.
Remember that when you use printf (for example) you never specify what stdout exactly is... That can be a file, a terminal, etc.
Basile Starynkevitch correctly explained:
After close(1); the file descriptor 1 (STDOUT_FILENO) is free. So creat("userlist",0644) is likely to re-use it…
This is because, as Jean-Baptiste Yunès wrote, "opens always returns the lowest free descriptor".
It should be stressed that the professor's code only likely works; it fails if file descriptor 0 is closed.
I would like to use Hadley Wickam's secure package from GitHub.
The example usage isn't explicit about how to create keys and where to store them and I'm messing something up (possibly more than one thing).
I installed the package
# install.packages("devtools")
devtools::install_github("s-u/PKI") # needed for bug fixes not currently on CRAN
devtools::install_github("hadley/secure")
set up a vault folder:
dir.create("vault")
Then the next step is to add a user / key:
secure::add_user("hackr", local_key())
and of course if I literally run that last line as-is it says
Error: No key matches id_rsa
Because I don't have a key. So, I used PuttyGen to create a public/private RSA key pair.
I saved them to my desktop and tried putting the full path in the command above:
secure::add_user("hackr", local_key("C:/Users/hackr/Desktop/r_public_key"))
But that didn't work:
Error: No key matches
Then I tried saving the public key in the vault and doing:
secure::add_user("hackr", local_key("r_public_key"))
but I got the same error. Next I tried putting the public key in the working directory (one directory higher than the vault) but got the same error.
Finally, I tried copying the keys to C:\Users\hackr\.ssh but that also led to the same error.
I suspect I need to save the key somewhere special (in Windows I'm not sure where that would be?) and/or I am using the wrong type of key since PuttyGen is for SSH (?).
It looks like local_key is assuming your key is stored in ~/.ssh (which is a reasonable assumption). By default it assumes that the file is named id_rsa.pub so if you've renamed it then you'll need to pass the name into local_key.
I haven't used this package but always remember those wise words "Hack-R view the source"
The issue is that Hadley's local_key() function is assuming your key is stored in ~/.ssh which is where the commands below will place it by default, and name it id_rsa.pub. If you have a different setup, you can change the defaults, or you could simply follow the steps below.
Step 1
Go to https://help.github.com/articles/generating-an-ssh-key/
Read up. It's useful stuff to know.
It will tell you to do this in the console:
ssh-keygen -t rsa -b 4096 -C "your_email#example.com"
Set a passphrase. Remember it.
Then enter this:
ssh-add ~/.ssh/id_rsa
Enter your passphrase.
Step 2
Your secure::add_user("hackr", local_key()) should work now.
I am trying to use lua to access redis values from nginx. When i execute lua files on command line there everything is ok i am able to read and write values to redis. But i when try to execute the same files from nginx by accessing a location in which access_by_lua directive is written the following error logged in error log file
no field package.preload['socket']
no file '/home/sivag/redis/redis-lua/src/socket.lua'
no file 'src/socket.lua'
no file '/home/sivag/lua/socket.lua'
no file '/opt/openresty/lualib/socket.so'
no file './socket.so'
no file '/usr/local/lib/lua/5.1/socket.so'
no file '/opt/openresty/luajit/lib/lua/5.1/socket.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
What is the reason for this and how can i resolve this?
In my case I just needed to install the lua-socket package, as the socket library is not built into the default Lua installation like it is in some other languages.
You get this error because your code executes the command require("socket")
This command will search for a file with that name in several directories. If successful the content will be executed as Lua code. If it is not successful you'll end up with your error message.
In order to fix this you have to add the path containing the file either to the system variable LUA_PATH or you have to add it to the global table package.path befor you require the file.
Lua will replace ? with the name you give to require()
For example
package.path = package.path .. ";" .. thisPathContainsTheLuaFile .. "?.lua"
Please read:
http://www.lua.org/manual/5.3/manual.html#pdf-require
https://www.lua.org/pil/8.1.html
I am trying to get a large dataset (3+ GB) from the following server:
ftp://podaac-ftp.jpl.nasa.gov/allData/ghrsst/data/L4/GLOB/JPL/MUR
I know RCurl is a good package for getting data from FTP. The file is a compressed netcdf file. I need to uncompress it to read it into R using ncdf4. It's compressed as bz2.
Importantly, the file is larger than I want on my hard drive, so saving a copy locally is not an ideal option. How can I access data on the file without saving a copy to my disk first?
Here's my attempt so far:
library(RCurl); library(ncdf4)
d = getURL('ftp://podaac-ftp.jpl.nasa.gov/allData/ghrsst/data/L4/GLOB/JPL/MUR/2015/144/20150524-JPL-L4UHfnd-GLOB-v01-fv04-MUR.nc.bz2')
d = bzfile(d, open = 'r')
d = nc_open(d)
But I'm stuck at this cryptic error after the first line:
Error in curlPerform(curl = curl, .opts = opts, .encoding = .encoding) :
embedded nul in string: 'BZh91AY&SY¦ÁÀÉ\0033[ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿáåÏ\035\017)³îÎ\u009dÍØcn]sw7½ÎkÜÞõï=uÎׯv]ìçn\u009dÎn½îê·±Þìê÷wSM\u008có·+ÎçW¹Ý=Ù×¹\u009cγÜëÞs½ÛN¹²w;\u009buÍÝ]{·k^çuªnìº-³6«[+Üå;\033m»Û½ow:w¹ïo{uyîî\u00937¬\\Ƶl¶½\u009dÖVìç¯{ÎõïoSm]Ý×\u009eî\u008dæî®î®î\vÛÕïgW\036î®wqîÝ\\ïw«6½Þï\036Ýrë§=¬Fg·\\íåÔÙº÷gu·3\u009bKmÛ\027Þ»\u0092îî\016îêwwm»\u009b·s;MÞÁ½½ÎóÍso^»q¯o;k\033iµ\u009bÛuyÝÞní5w:ï]ÓuÎo[«\033:åÞvEÜíÎç½ÝË\u009eìQNöÔ\u008e\u0094vmÝȯg»e lÍ^\u008a©'
It seems to be an encoding issue based on other similar problems but I tried both .encoding = 'UTF-8' and .encoding = 'ISO-8859-1' as shown in the getURL() documentation but neither work.
I've seen other answers to problems like this but they all seem to involve editing the source file. I don't have write access to this file, however. Any help?
I'd use httr for this
library("httr")
library("ncdf4")
url <- 'ftp://podaac-ftp.jpl.nasa.gov/allData/ghrsst/data/L4/GLOB/JPL/MUR/2015/144/20150524-JPL-L4UHfnd-GLOB-v01-fv04-MUR.nc.bz2'
res <- GET(url, write_disk(basename(url)))
# uncompress - I used OSX's default compression tool
nc_open(sub("\\.bz2", "", res$request$output$path))
the only step i didn't sort out programatically is un-compressing the bz2 file, just did that with OSX's default tool
I don't know much at all about R, but you should be able to do this with curl in FTP mode by changing the output to stdout rather than a local filename and then using bz2 to uncompress the file you want from its standard input.
So, for example, I can do this:
curl --output - --user user:password 'ftp://127.0.0.1/somefile.bz2' | bz2 ...
Maybe you can start that from within R? Or make a fifo with:
mkfifo fifo
curl ....
and then read from the file called fifo in R.
Or maybe R has a system() command, and you could do:
system('mkfifo fifo; curl ..... | bz2 .... > fifo &')
and then read from the file called fifo in R.
I have a UNIX application written in ansi C that writes data directly to a file. This file is specified by one of the argument parameters.
For testing purposes, I can use /dev/null for the filename, which effectively redirects the output to nothing.
I would like to be able to redirect the output to stdout by a similar method. Is this possible? If so, how? I've tried the following with no luck:
a.out -f /dev/ttys000
(where /dev/ttys000 was the tty specified by a 'w' listing)
/dev/stdout
You could detect the string "stdout" argument and then use the stdout filehandle in C (1)
http://en.wikipedia.org/wiki/File_descriptor
Or use /dev/stdout or /dev/fd/1
If this is a 'built-in' feature rather than a temporary thing for testing, you might want to use the C functions on the stdout file descriptor rather than the device node as the C standard is a bit more hardy than the POSIX standard imho.