I am trying to use pyghsheets from within a Jupyter notebook and I do not get it to work, while the same piece of code works nicely from within ipython.
from pathlib import Path
import pygsheets
creds = Path(r"/path/to/client_secret.json")
gc = pygsheets.authorize(client_secret=creds)
book = gc.open_by_key("__key__of__sheet__")
wks = book.worksheet_by_title("Sheet1")
wks.clear(start="A2")
When called from within ipython everthing works fine, whereas from within a Jupyter notebook I get
RefreshError: ('invalid_grant: Token has been expired or revoked.', {'error': 'invalid_grant', 'error_description': 'Token has been expired or revoked.'})
I run both pieces from within the same conda environment. Any suggestion on how to narrow down the problem (and solutions) are very welcome!
It turns out that current working directory of my Jupyter notebook was not the same as the one of my plain ipython. Now pygsheets stores the token that is used for authentication with Google in the current working directory. If the .json file in that directory is invalid, authentication will fail.
You can add a parameter credentials_directory=... to specify a previously validated token file.
The solution I ended up with was
gc = pygsheets.authorize(client_secret=creds, credentials_directory=creds.parent)
That way token and credential files are in the same directory.
Related
I'm using the gmailr package for sending emails from a r script.
Locally it's all working fine, but when I try to run this during a docker build step in google cloud I'm getting an error.
I implemented it in the following way described here.
So basically, locally the part of my code for sending emails looks like this:
gm_auth_configure(path = "credentials.json")
gm_auth(email = TRUE, cache = "secret")
gm_send_message(buy_email)
Please note, that I renamed the .secret folder to secret, because I want to deploy my script with docker in gcloud and didn't want to get any unexpected errors due to the dot in the folder name.
This is the code, which I'm now trying to run in the cloud:
setwd("/home/rstudio/")
gm_auth_configure(path = "credentials.json")
options(
gargle_oauth_cache = "secret",
gargle_oauth_email = "email.which.was.used.to.get.secret#gmail.com"
)
gm_auth(email = "email.which.was.used.to.get.secret#gmail.com")
When running this code in a docker build process, I'm receiving the following error:
Error in gmailr_POST(c("messages", "send"), user_id, class = "gmail_message", :
Gmail API error: 403
Request had insufficient authentication scopes.
Calls: gm_send_message -> gmailr_POST -> gmailr_query
I can reproduce the error locally, when I do not check the
following box.
Therefore my first assumption is, that the secret folder is not beeing pushed correctly in the docker build process and that the authentication tries to authenticate again, but in a non interactive-session the box can't be checked and the error is thrown.
This is the part of the Dockerfile.txt, where I'm pushing the files and running the script:
#2 ADD FILES TO LOCAL
COPY . /home/rstudio/
WORKDIR /home/rstudio
#3 RUN R SCRIPT
CMD Rscript /home/rstudio/run_script.R
and this is the folder, which contains all files / folders beeing pushed to the cloud.
My second assumption is, that I have to somehow specify the scope to use google platform for my docker image, but unfortunately I'm no sure where to do that.
I'd really appreciate any help! Thanks in advance!
For anyone experiencing the same problem, I was finally able to find a solution.
The problem is that GCE auth is set by the "gargle" package, instead of using the "normal user OAuth flow".
To temporarily disable GCE auth, I'm using the following piece of code now:
library(gargle)
cred_funs_clear()
cred_funs_add(credentials_user_oauth2 = credentials_user_oauth2)
gm_auth_configure(path = "credentials.json")
options(
gargle_oauth_cache = "secret",
gargle_oauth_email = "sp500tr.cloud#gmail.com"
)
gm_auth(email = "email.which.was.used.for.credentials.com")
cred_funs_set_default()
For further references see also here.
Hello I'm trying to do web scraping with the python module requests-html to handle dynamic content on the page https://www.monster.com/jobs/search?q=Software+Engineer&where=. My code is:
from requests_html import HTMLSession
url = 'https://www.monster.com/jobs/search?q=Software+Engineer&where='
session = HTMLSession()
response = session.get(url)
response.html.render()
but when I run response.html.render() I get this error
OSError: [WinError 14001] The application has failed to start because its side-by-side configuration is incorrect. Please see the application event log or use the command-line sxstrace.exe tool for more detail
The first time I ran render() I got
[W:pyppeteer.chromium_downloader] start chromium download.
Download may take a few minutes.
[W:pyppeteer.chromium_downloader]
chromium download done.
[W:pyppeteer.chromium_downloader] chromium extracted to: C:\Users\user\AppData\Local\pyppeteer\pyppeteer\local-chromium\588429
however the file path doesn't exist but pyppeteer is actually an installed package (pyppeteer==0.2.5). Does anyone have an idea what is going on?
You're having this issue because chromium setup failed.
You can either try to reinstall request_html or what I did was switching from the python from the Windows store to the download from the python website and then installing request_html again.
After having everything setup correctly with the downloaded python I switched back to python 3.9 from the store and everything is still working.
I am trying to set up a Google Co-lab notebook that runs in R and can read a GCS bucket from a GCP project. I am using the googleCloudStorageR package. To authenticate and read the bucket, the initial Co-lab notebook runs the following Python commands:
!gcloud auth login
!gcloud config set project project_name
!gcloud sql instances describe project_name
How can I run the above commands in R using the googleCloudStorageR package ? In the documentation for the package, they mention using the gcs_auth function that reads an authentication JSON file. However, since I will be accessing the buckets through a Co-Lab notebook running on R, I do not want to use an authentication file and instead want to authenticate and connect to the GCP storage in real-time from the Co-Lab notebook. Thank you!
Figured this out. In a Co-lab notebook, run the following code snippet:
install.packages("httr")
install.packages("R.utils")
install.packages("googleCloudStorageR")
if (file.exists("/usr/local/lib/python3.6/dist-packages/google/colab/_ipython.py")) {
library(R.utils)
library(httr)
reassignInPackage("is_interactive", pkgName = "httr", function() return(TRUE))
}
library(googleCloudStorageR)
options(
rlang_interactive = TRUE,
gargle_oauth_email = "email_address",
gargle_oauth_cache = TRUE
)
token <- gargle::token_fetch(scopes = "https://www.googleapis.com/auth/cloud-platform")
googleAuthR::gar_auth(token = token)
There is an issue with gargle authentication that the googleCloudStorageR package uses. A workaround that is similar to the one listed here (https://github.com/r-lib/gargle/issues/140) is to generate a token for cloud scopes, which would give us a token object that we would then use in the gar_auth function.
I have just upgraded Jupyter to the version 4.3.1
While I can open previously created ipynb files, I cannot create new ones.
When I try to create a new notebook file, I get a pop up windows saying:
Creating Notebook Failed
An error occurred while creating a new notebook
Forbidden
In the terminal I notice this output:
[W 12:53:23.375 NotebookApp] 403 POST /api/contents (::1): '_xsrf' argument missing from POST
[W 12:53:23.383 NotebookApp] 403 POST /api/contents (::1) 8.92ms referer=http://localhost:8888/tree?token=e7fbbb58516dc1359fcc26a1079093166a1f713ee5b94ccd
I use Jupyter with Python 3.5.2 and IPython 5.1.0
Another alternative to confirm the issue is to open your Jupyter Session in another browser and you might be redirected to a screen like the following:
If you open a new console and type
jupyter notebook list
You'll see your current notebook and the URL will contain a token. Open that in a new tab and problem solved.
Output command should look like this:
Currently running servers:
http://localhost:8888/?token=cbad1a6ce77ae284725a5e43a7db48f2e9bf3b6458e577bb :: <path to notebook>
I had to enable cookies in the browser (which I had intentionally disabled). Then the "Forbidden" error disappeared, everything is OK now.
The generally accepted solution to prevent XSRF is to cookie every user with an unpredictable value and include that value as an additional argument with every form submission on your site.
From: http://tornado.readthedocs.io/en/latest/guide/security.html#cross-site-request-forgery-protection
Jupyter blocks non-local requests. To access Jupyter from an external address we can execute it with the following parameters:
jupyter notebook --NotebookApp.allow_origin=* --NotebookApp.allow_remote_access=1
I had this problem just now, but I noticed that it worked in Edge. Deleting all browser cache including cookies in Chrome solved this in my case.
I need some Powershell advice.
I need to install an application's MSP update file on multiple Win08r2 servers. If I run these commands locally, within the target machine's PS window, it does exactly what I want it to:
$command = 'msiexec.exe /p "c:\test\My Application Update 01.msp" REBOOTPROMPT=S /qb!'
invoke-wmimethod -path win32_process -name create -argumentlist $command
The file being executed is located on the target machine
If I remotely connect to the machine, and execute the two commands, it opens two x64 msiexec.exe process, and one msiexec.exe *32 process, and just sits there.
If I restart the server, it doesn't show that the update was installed, so I don't think it's a timing thing.
I've tried creating and remotely executing a PS1 file with the two lines, but that seems to do the same thing.
If anyone has advice on getting my MSP update installed remotely, I'd be all ears.
I think I've included all the information I have, but if something is missing, please ask questions, and I'll fill in any blanks.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
My process for this is:
Read a CSV for server name and Administrator password
Create a credential with the password
Create a new session using the machine name and credential
Create a temporary folder to hold my update MSP file
Call a PS1 file that downloads the update file to the target server
>>> Creates a new System.Net.WebClient object
>>> Uses that web client object to download from the source to the location on the target server
Call another PS1 file that applies the patch that was just downloaded –>> This is where I’m having issues.
>>> Set the variable shown above
>>> Execute the file specified in the variable
Close the session to the target server
Move to the next server in the CSV…
If I open a PS window and manually set the variable, then execute it (as shown above in the two lines of code), it works fine. If I create a PS1 file on the target server, containing the same two lines of code, then right click > ‘Run With PowerShell’ it works as expected / desired. If I remotely execute my code in PowerGUI, it returns a block of text that looks like this, then just sits there. RDP’d into the server, the installer never launches. My understanding of the “Return Value” value is that “0″ means the command was successful.
PSComputerName : xx.xx.xx.xx
RunspaceId : bf6f4a39-2338-4996-b75b-bjf5ef01ecaa
PSShowComputerName : True
__GENUS : 2
__CLASS : __PARAMETERS
__SUPERCLASS :
__DYNASTY : __PARAMETERS
__RELPATH :
__PROPERTY_COUNT : 2
__DERIVATION : {}
__SERVER :
__NAMESPACE :
__PATH :
ProcessId : 4808
ReturnValue : 0
I even added a line of code between the variable and the execution that creates a text file on the desktop, just to verify I was getting into my ‘executeFile’ file, and that text file does get created. It seems that it’s just not remotely executing my MSP.
Thank you in advance for your assistance!
Catt11.
Here's the strategy I used to embed an msp into a powershell script. It works perfectly for me.
$file = "z:\software\AcrobatUpdate.msp"
$silentArgs = "/passive"
$additionalInstallArgs = ""
Write-Debug "Running msiexec.exe /update $file $silentArgs"
$msiArgs = "/update `"$file`""
$msiArgs = "$msiArgs $silentArgs $additionalInstallArgs"
Start-Process -FilePath msiexec -ArgumentList $msiArgs -Wait
You probably don't need to use the variables if you don't want to, you could hardcode the values. I have this set up as a function to which I pass those arguments, but if this is more of a one-shot deal, it might be easier to hard-code the values.
Hope that helps!
using Start-Process for MSP package is not a good practice because some update package lockdown powershell libs and so you must use WMI call