I am struggling with writing a script that would somehow scrape the https://www.rstudio.com/products/rstudio/download/ for the number of the latest RStudio version, download it and install it.
Since I am an R programmer, I started to write an R script using rvest package. I managed to scrape the download link for the RStudio server, but I still cannot get the RStudio itself.
Here is the R code for getting a download link for the 64bit RStudio-server for Ubuntu.
if(!require('stringr')) install.packages('stringr', Ncpus=8, repos='http://cran.us.r-project.org')
if(!require('rvest')) install.packages('rvest', Ncpus=8, repos='http://cran.us.r-project.org')
xpath<-'//code[(((count(preceding-sibling::*) + 1) = 3) and parent::*)]'
url<-'https://www.rstudio.com/products/rstudio/download-server/'
thepage<-xml2::read_html(url)
the_links_html <- rvest::html_nodes(thepage,xpath=xpath)
the_links <- rvest::html_text(the_links_html)
the_link <- the_links[stringr::str_detect(the_links, '-amd64\\\\.deb')]
the_r_uri<-stringr::str_match(the_link, 'https://.*$')
cat(the_r_uri)
Unfortunately, the RStudio desktop download page has completely different layout, and I the same approach doesn't work here.
Can someone help me with this? I can't believe, that all the data scientist in the world manually upgrade their RStudio!
There is an even simpler version of the script, that reads the version of the RStudio-server. Bash version:
RSTUDIO_LATEST=$(wget --no-check-certificate -qO- https://s3.amazonaws.com/rstudio-server/current.ver)
or R version:
scan('https://s3.amazonaws.com/rstudio-server/current.ver', what = character(0))
But the version of the RStudio-desktop still eludes me.
It seems that you can get the latest stable version number from the url http://download1.rstudio.org/current.ver and it is more up to date (for some unknown reason), at least at the time of writing this answer.
$ curl -s http://download1.rstudio.org/current.ver
1.1.447
$ curl -s https://www.rstudio.org/links/check_for_update?version=1.0.0 | grep -oEi 'update-version=([0-9]+\.[0-9]+\.[0-9]+)' | awk -F= '{print $2}'
1.1.423
Found that here: https://github.com/yutannihilation/ansible-playbook-r/blob/master/tasks/install-rstudio-server.yml
If you query RStudio's check_for_update with a version string you'll get back the update version and the URL of where to get it from:
https://www.rstudio.org/links/check_for_update?version=1.0.0
update-version=1.0.153&update-url=https%3A%2F%2Fwww.rstudio.com%2Fproducts%2Frstudio%2Fdownload%2F&update-message=RStudio%201.0.153%20is%20now%20available%20%28you%27re%20using%201.0.0%29&update-urgent=0
See here:
https://github.com/rstudio/rstudio/blob/54cd3abcfc58837b433464c793fe9b03a87f0bb4/src/cpp/session/modules/SessionUpdates.R
If you really want to scrape it from the download page then I'd get the href of the <a> in the first <td> of the first <table> of class "downloads", and then parse out the three dot-separated numbers between "RStudio-" and ".exe". RStudio release versions over all platforms so getting it from the Windows download should be sufficient.
> url = "https://www.rstudio.com/products/rstudio/download/"
> thepage<-xml2::read_html(url)
> html_node(thepage, ".downloads td a") %>% html_attr("href")
[1] "https://download1.rstudio.org/RStudio-1.0.153.exe"
There's a nearly-solution here:
https://hub.docker.com/r/rocker/rstudio-daily/~/dockerfile/
In this script, which scrapes for the latest builds:
https://raw.githubusercontent.com/rocker-org/rstudio-daily/master/latest.R
You'll want to modify that script to be more strict about what it accepts, i.e. I would want this one rstudio-server-1.1.355-amd64.deb and not the stretch variant.
(But you can modify it to target the kind of build you want anyway, this is the daily builds, RStudio Server for Ubuntu.)
If anyone is interested, here is my ultimate RServer-desktop-on-Ubuntu update script. It installs RStudio-desktop 64bit and then, if Fira Console font is available, applies a patch from https://github.com/tonsky/FiraCode/wiki/RStudio-instructions for the RStudio, so the ligatures start working.
#!/bin/bash
if dpkg -s rstudio >/dev/null 2>/dev/null; then
ver=$(apt show rstudio | grep Version)
pattern='^Version: ([0-9.]+)\s*$'
if [[ $ver =~ $pattern ]]; then
ourversion=${BASH_REMATCH[1]}
netversion=$(Rscript -e 'cat(stringr::str_match(scan("https://www.rstudio.org/links/check_for_update?version=1.0.0", what = character(0), quiet=TRUE), "^[^=]+=([^\\&]+)\\&.*")[[2]])')
if [[ $ourversion != $netversion ]]; then
RSTUDIO_URI=$(Rscript /tmp/get_rstudio_uri.R)
fi
tee /tmp/get_rstudio_uri.R <<EOF
if(!require('rvest')) install.packages('rvest', repos='http://cran.us.r-project.org')
xpath='.downloads:nth-child(2) tr:nth-child(5) a'
url = "https://www.rstudio.com/products/rstudio/download/"
thepage<-xml2::read_html(url)
cat(html_node(thepage, xpath) %>% html_attr("href"))
EOF
RSTUDIO_URI=$(Rscript /tmp/get_rstudio_uri.R)
wget -c --output-document /tmp/rstudio.deb $RSTUDIO_URI
sudo dpkg -i /tmp/rstudio.deb
rm /tmp/rstudio.deb
rm /tmp/get_rstudio_uri.R
if fc-list |grep -q FiraCode; then
if !grep -q "text-rendering:" /usr/lib/rstudio/www/index.htm; then
sudo sed -i '/<head>/a<style>*{text-rendering: optimizeLegibility;}<\/style>' /usr/lib/rstudio/www/index.htm
fi
fi
fi
fi
I would like to automate a build - for now, during my development, so no security stuff involved.
I have created a script that moves libs to /usr/local/lib and issues ldd command.
These things require sudo.
Running the script from the builder (Qt Creator), I am not prompted to enter my sudo password, and I get the error
sudo: no tty present and no askpass program specified
Sorry, try again.
I have found a few solutions to this but it just did not work... what am I missing ?
Exact code:
in myLib.pro
#temporary to make my life easier
QMAKE_POST_LINK = /home/me/move_libs_script
in move_libs_script:
#!/bin/bash
sudo cp $HOME/myLib/myLib.so.1 /usr/local/lib/
sudo ldconfig
I did as suggested by the answer above: edited visudo and added the script... even added qmake...
sudo visudo
added at the end:
me ALL=NOPASSWD: /home/me/move_libs_script, /usr/bin/qmake-qt4
It saved file: /etc/sudoers.tmp (and doing the command sudo visudo again I saw that my changes were kept so I am not sure what is with the tmp)
Still same errors
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: 3 incorrect password attempts
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: 3 incorrect password attempts
Edit: after asking the question I found a suggested similar question:
https://stackoverflow.com/a/10668693/1217150
So I tried to add a custom step...
Result:
09:50:03: Running build steps for project myLib...
09:50:03: Could not start process "ssh-askpass Sudo Password | sudo -S bash ./move_libs_script"
Error while building project myLib (target: Desktop)
When executing build step 'Custom Process Step'
(if I run from terminal I get asked for password)
New edit: so I thought I can outsmart the system and call a script that calls my script...
myLib.pro
QMAKE_POST_LINK = /home/me/sudo_move_libs_script
sudo_move_libs_script:
#!/bin/bash
ssh-askpass Sudo Password | sudo -S bash $HOME/move_libs_script
got it !!! I will post as answer i guess
New edit as answer to comment:
in mainExe.pro:
QMAKE_POST_LINK = ./link_lib
in link_lib:
#!/bin/sh
LD_LIBRARY_PATH=$HOME/myLib
export LD_LIBRARY_PATH
Result: Executable fails because the lib is not to be found (of course before testing I emoved the copy from /usr/local/lib)
Solution 1:
To run the command I needed with sudo from Qt, and be asked for password
myLib.pro
QMAKE_POST_LINK = /home/me/sudo_move_libs_script
sudo_move_libs_script:
#!/bin/bash
ssh-askpass Sudo Password | sudo -S bash $HOME/move_libs_script
move_libs_script:
#!/bin/bash
cp $HOME/myLib/myLib.so.1 /usr/local/lib/
ldconfig
Solution 2
Avoid using sudo: run a custom executable from Qt Creator (just like I would when project is deployed) - I can execute my program with correct dependency. Unfortunately it does not work for debug.
In the Run Configuration, place a script "dummy_executable" instead the app executable:
dummy_executable:
#!/bin/sh
LD_LIBRARY_PATH=$HOME/myLib
export LD_LIBRARY_PATH
$HOME/myExec/myExec "$#
This method is great for running the program ... unfortunately gdb fails... I think it tries to debug the shell script ? So it won't work if I try to step through my program.
Choosing Solution 1, because it has advantages: allows me to build and debug the program, which is what I really need, and with the dependencies set I get the library built before running the program - which is perfect.
Thank you Etan Reisner
I'm not following how the environment variable $NLSPATH value is being cleared/reset when running gmake. In my bash shell, if I issue echo $NLSPATH, I see the expected /usr/lib/nls/msg/%L/%N: (etc).
I then create a Makefile like this:
all:
echo $$NLSPATH
echo $$PATH
And running gmake all gives me
echo $NLSPATH
echo $PATH
/usr/bin:/etc:(etc as expected)
I also tried -e with the same result;
gmake -e all
echo $NLSPATH
echo $PATH
/usr/bin:/etc:(etc as expected)
I've looked at /etc/environment and /etc/.profile and $NLSPATH is set correctly there.
Also if I run the standard AIX make, the NLSPATH is shown as expected. So this seems to be gmake and/or the way it invokes the current shell.
Could someone suggest where I should be looking ? [EDIT]
As I'm new, I can't hit answer right away...
Finally found it - The following technote describes it (albeit briefly) as a security limitation;
http://www-01.ibm.com/support/docview.wss?uid=swg21516415
Issuing;
cd /opt/freeware/bin
sudo chmod -s make
solves this issue for me although I'm left wondering there is some aspect of gmake that may later expect the setuid flag.
IIRC GNU make on AIX needs the setgid flag in order to enable the -l flag, because on this OS you can't retrieve the load average of the system as a normal user (no idea why that is considered more of a security issue than having any program that needs to check the load average be setgid, but... no one asked me).
As long as you don't use -l when running GNU make, you won't have any problem.
I created a batch file for windows that executes some xmlstarlet commands. I want to write it as .sh file so that i can run it on mac. The problem is.. Some commands are working fine in windows but not in mac. It didn't show any error too. Eg.
**xml ed -L -d //intent-filter//category[#android:name='android.intent.category.LAUNCHER'] my_folder\AndroidManifest.xml**
In windows, above command deletes the mentioned xml tag. BUt it does nothing in mac.
But the command
**xml sel -t -m //manifest -v //manifest/#package mim_apk_proj\AndroidManifest.xml**
is working fine in both mac and windows.
I have installed xml tool. Checked /usr/local/bin. It has libxslt.dylib and libxml2.dylib. I dont know where the problem lies?
Can someone help?
The quoting rules for bash (that's the shell on your mac, right?) are different from cmd.exe (the Windows shell), in particular, cmd.exe treats ' as a normal character while to bash it is a quoting character so it isn't passed to the program. In bash you therefore need to quote the 's as well:
xml ed -L -d //intent-filter//category[#android:name='android.intent.category.LAUNCHER'] my_folder\AndroidManifest.xml
# becomes
xml ed -L -d "//intent-filter//category[#android:name='android.intent.category.LAUNCHER'] my_folder\AndroidManifest.xml"
# or, since XPath treats both kinds of quotes identically you can also use
xml ed -L -d '//intent-filter//category[#android:name="android.intent.category.LAUNCHER"] my_folder\AndroidManifest.xml'
The second fix is safer because it also prevents bash from doing any variable expansion if you use $, but the first fix has the advantage of working in Windows as well.
I am trying
sudo mount -t cifs //<server>/<share> -o username=user#domain,password=**** /mnt/<mountpoint>
error message:
mount: wrong fs type, bad option, bad superblock on //server/share,
missing codepage or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
The syslog has
CIFS VFS: cifs_mount failed w/return code = -22
I am able to mount the same share on another centos system. I can ping the server, mount point directory has been created.
I ran into this problem when using a host name and solved it by using an IP address. E.g.:
use
mount -t cifs //192.168.1.15/share
rather than
mount -t cifs //servername/share
Another possible solution is to install
cifs-utils
.
Ah, the dreaded -22. Basically this seems to be used as a catchall for "something didn't work", although technically it's referred to as an invalid argument.
The client does IMHO a very poor job of telling you the actual problem. (This may not be its fault - it doesn't always have access to that information).
However -- have you checked the logs on the server/machine you are connecting to?
I was connecting to an OS X samba server, and learned from what I found in the logs there that it was necessary to specify additional options under -o as follows:
nounix,sec=ntlmssp
Among the things these settings enable are "allow long names", and "ignore UNIX filename endings"...sec is to specify security flags.
Another possibility is that you're trying to access a filesystem of a type that mount.cifs can't actually handle.
For RHEL/Centos install package - "cifs-utils"
Maybe move the target?
sudo mount -t cifs -o username=user#domain,password=**** //<server>/<share> /mnt/<mountpoint>
Or maybe this solution? (Ubuntu, Debian methods)
sudo apt-get install smbfs
Or for CentOS, RedHat, Fedora try:
sudo yum install samba-client
I had a similar issue on Ubuntu 12.04 with the "mount" package (version 2.20.1-1ubuntu3).
It happened when I was trying to mount the server share using its hostname rahter than its IP.
Another way to solve the issue on Ubuntu was to install the cifs-utils package. That way I could also mount the samba share using the exact same command line (or fstab) but with hostname.
sudo mount -t cifs //hostname/share -o username=user,password=pwd /mnt/share
Just did a clean install of Ubuntu 12.04 LTS and got this trying to hook up my Linux HTPC.
Solved it by running: sudo apt-get install cifs-utils then remounting it.
CIFS returns code "-22" in many cases (not only invalid arguments).
For me installing keyutils did the trick:
apt-get install keyutils
My distribution is "Ubuntu 14.04.2 LTS".
I figured this out by increasing the logging verbosity of CIFS:
echo 7 > /proc/fs/cifs/cifsFYI
# disable again via:
#echo 0 > /proc/fs/cifs/cifsFYI
Documentation on the bitmask ("7") for cifsFYI can be found here: https://www.kernel.org/doc/readme/Documentation-filesystems-cifs-README
After trying to mount once more dmesg included more helpful information:
Dec 7 12:34:20 pc1471 kernel: [ 5442.667417] CIFS VFS: dns_resolve_server_name_to_ip: unable to resolve:
Another maybe helpful link:
http://vlkan.com/blog/post/2015/01/08/smb-mount-troubleshoot/
I have Ubuntu Server 12.10 x64 installed as a VMware VM, running on OS X 10.8 (Mountain Lion).
On the Mac, in SYSTEM PREFERENCES > SHARING > FILE SHARING (on), I added a folder to share. For my tests, I created a new folder within my Public folder called "ubuntu".
In Ubuntu, I issued the following commands:
sudo mkdir /media/target
sudo mount.cifs //10.0.20.3/ubuntu /media/target -o username=davidallie,nounix,sec=ntlmssp,rw
Ubuntu prompted me for the password and, once entered, mounted the folder. I then ran:
df -H
which allowed me to verify the mounts and mount-points.
This has recently manifested thanks to a kernel bug in v5.18.8+, I was able to reproduce on v5.18.9 and v5.18.11.
Here is the relevant ticket on kernel.org, quote:
it appears that kernel 5.18.8 breaks cifs mounts on my machine. With
5.18.7, everything works fine. With 5.18.8, I am getting:
$ sudo mount /mnt/openmediavault/
mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs) and kernel
log messages (dmesg)
The relevant /etc/fstab line is:
//odroidxu4.local/julian /mnt/openmediavault cifs
credentials=/home/julas/.credentials,uid=julas,gid=julas,vers=3.1.1,nobrl,_netdev,auto
0 0
Here is the offending commit, and here is the fix, which applies cleanly to v5.18.11. The cause is, from what I understand, a bug in old versions of the samba server in the negotiation protocol.
If this is your issue, you can:
patch your kernel yourself;
downgrade to v5.18.7;
switch to an LTS kernel;
use the userspace (and also really slow and awful) gvfs-smb;
upgrade the samba version on your server; or
add vers=2.0 to the mount.cifs options in /etc/fstab.
Note that while I haven't tried the last one personally, the venerable #SEBiGEM has confirmed in the comments that it works for v5.18.10.
Note also that I didn't try upgrading samba on the server at all because I hate touching the box it's running on - every time I upgrade anything everything breaks. Doing so might also not be an option for those with NAS appliances.
As a personal sidenote, it's a little sad that so many different things can cause -22. My answer is correct, but very very niche and specific to this point in time. I imagine in a month it will simply be useless noise.
Just experience the problem on RHEL 5. You don't need to install the samba suite, just the samba-client and any dependencies.
Maybe it's too late, but simplest solution described in kernel bug 50631:
in the latest code, unc mount parameter in mandatory. Modified command works for me:
sudo mount -t cifs //<server>/<share> -o username=user#domain,password=****,unc=\\\\<server>\\<share> /mnt/<mountpoint>
Try run the comamnd:
$modinfo cifs
filename: /lib/modules/3.2.0-60-virtual/kernel/fs/cifs/cifs.ko
version: 1.76
description: VFS to access servers complying with the SNIA CIFS Specification e.g. Samba and Windows
license: GPL
author: Steve French <sfrench#us.ibm.com>
srcversion: 9435BBC2F61D29F06643803
depends:
intree: Y
vermagic: 3.2.0-60-virtual SMP mod_unload modversions 686
parm: CIFSMaxBufSize:Network buffer size (not including header). Default: 16384 Range: 8192 to 130048 (int)
parm: cifs_min_rcv:Network buffers in pool. Default: 4 Range: 1 to 64 (int)
parm: cifs_min_small:Small network buffers in pool. Default: 30 Range: 2 to 256 (int)
parm: cifs_max_pending:Simultaneous requests to server. Default: 32767 Range: 2 to 32767. (int)
parm: echo_retries:Number of echo attempts before giving up and reconnecting server. Default: 5. 0 means never reconnect. (ushort)
parm: enable_oplocks:Enable or disable oplocks (bool). Default:y/Y/1 (bool)
If your getting any error then cifs is not installed. Just check with your admin. I thought it helps out.
Adding the option vers=3.0 to the mount command worked for me: sudo mount -t cifs -v <src> <dst> -o ...,vers=3.0,...
You need to install cifs-utils first , just as follows:
sudo yum install cifs-utils
I know this is old, but on older cifs-utils versions, you may have to add the following two lines to /etc/request-key.conf
create cifs.spnego * * /usr/sbin/cifs.upcall -c %k
create dns_resolver * * /usr/sbin/cifs.upcall %k
Workaround without installing additional packages (cifs-utils adds another 81mb in Debian Stretch):
$ FILESERVER_IP=$(getent hosts myfileserver.com | awk '{ print $1 ; exit }')
$ sudo mount -t cifs //${FILESERVER_IP}/<share> -o username=user#domain,password=**** /mnt/<mountpoint>
Many answers, but wasn't work for me.
Solution:
My NAS didn't support Samba 3.0, on which my mount switch automatically.
So I downgraded smb version:
mount -t cifs //192.168.0.2/Share -o rw,vers=1.0,username=*****,password=******* /media/1
It's work.