Run Azure Notebook By API (external) - jupyter-notebook

I am trying to run the notebook from node, everything is working fine except the parameters are not accepted by the notebook instead it is sending the output based on default params. I am not getting where I am doing wrong.
Below is my call:
var job_payload = {
"run_name": runName,
"existing_cluster_id": 'cluster_id',
"notebook_task":
{
"notebook_path": notebookPath
},
"notebook_params": notebook_params //{'x':1,'y':2}
}
var url = "https://<location>.<azr_databricks>.net/api/2.0/jobs/runs/submit";
var options = {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': 'Bearer token'
},
body: JSON.stringify(job_payload),
};
My notebook:
import json
dbutils.widgets.text("x", '3', "firstParam")
dbutils.widgets.text("y", '4', "secondParam")
x=int(dbutils.widgets.get("x"))
y=int(dbutils.widgets.get("y"))
sum=x+y
class Output:
def __init__(self, val):
self.resultTest2 = val
p1 = Output(sum)
print(p1.resultTest2)
result=json.dumps(p1.__dict__)
#RETURNING THE OUTPUT
dbutils.notebook.exit(result)
I am sending x:1 and y:2 as param but instead of getting output 3 I am getting 7 which is default value.
As I am not getting much help from the documentation, please help:
Document URL: Microsoft link

I got the answer that where I was wrong from the below link :
StackOverflow Link
the job_payload will look like below:
var job_payload = {
"run_name": runName,
"existing_cluster_id": 'cluster_id',
"notebook_task":
{
"notebook_path": notebookPath,
"base_parameters":notebook_params //{'x':1,'y':2}
},
}
var url = "https://<location>.<azr_databricks>.net/api/2.0/jobs/runs/submit";
var options = {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': 'Bearer token'
},
body: JSON.stringify(job_payload),
};
Now, it is working fine.

Related

How do I change the JSON stream I get back from Fetch using Deno?

I am new to Deno and have the following simple code...
const getCOP = async()=>{
const resp = await fetch("....", {
headers: {
accept: "application/json",
apiKey
},
});
return await resp.body;
}
let resp = {}
return new Response(resp.body, {
status: resp.status,
headers: {
"content-type": "application/json",
},
});
resp.body = await getCOP();
resp.status = 200;
It returns
{
"success": true,
"timestamp": 1675621083,
"base": "COP",
"date": "2023-02-05",
"rates": {
"EUR": 0.000199,
"GBP": 0.000179,
"USD": 0.000216
}
}
What I would like to do would be the equivalent of this in normal JS...
return {
rate : resp.body.rates,
copPerDollar : 1 / resp.body.rates.USD
}
Of course this doesn't seem to work because instead of an actual json obj it is a readable stream. How do I transform this stream into a json object and then restream it to the body of the sent request?

Why doesn't my fetch POST function work in Svelte?

I have a Svelte application that is supposed to perform CRUD operations with a local JSON file via the Fetch API.
The "GET" operations works as intended but when I tried to create the "POST" function, I got the below error message:
Uncaught (in promise) Error: {#each} only iterates over array-like objects.
at validate_each_argument (index.mjs:1977)
at Object.update [as p] (index.svelte? [sm]:33)
at update (index.mjs:1057)
at flush (index.mjs:1025)
Below is the code in index.svelte:
<script>
import { onMount } from 'svelte';
let data1 = '';
onMount(async function () {
const data = await (await fetch('http://localhost:5000/data1')).json();
data1 = data;
console.log(data);
});
const createData1 = async () => {
const data =
(await fetch('http://localhost:5000/data1'),
{
method: 'Post',
body: JSON.stringify({
id: data1.length + 1,
text: '',
}),
headers: {
'Content-Type': 'application/json'
}
});
data1 = data;
console.log(data);
};
</script>
<div style="display: grid; place-items:center;">
<div class="horizontal">
{#each data1 as d1}
<div contenteditable="true">{d1.text}</div>
{/each}
<button type="submit" on:click|preventDefault={createData1}>+</button>
</div>
</div>
And below is the contents of the JSON file:
{
"data1": [
{
"id": 1,
"text": "blabla",
},
{
"id": 2,
"text": "bla bla",
}
]
}
Why isn't the the object being created? It is inside an array after all.
As your log shows, data is an object, not an array. data.captions is the array you want to iterate over.
So you'll want to slightly modify your createData method near the end:
const createData1 = async () => {
...
// data1 = data;
data1 = data.data1;
console.log(data);
};
It looks to me like you need to adjust your code syntax a bit and make sure you convert your response to JSON and you will be in business. This should work:
const data = (await fetch('http://localhost:5000/data1', {
method: "POST",
body: JSON.stringify({
id: data1.length + 1,
text: ''
}),
headers: {
"content-type": "application/json"
}
})).json();
data1 = data;
The notable adjustments are:
Use the fetch overload that takes first argument as URL and the second argument as the request options object (Looks like this is what you were after and an extra "(" character was throwing it of)
Make sure to convert the result of the POST call to JSON. Your "GET" request does this conversion and you get what you are expecting.

Vue 3: Cannot read property 'id' of null

I want to show list product from api but it shows the error:
Uncaught (in promise) TypeError: Cannot read property 'id' of null
at eval (Home.vue?bb51:103)
at renderList (runtime-core.esm-bundler.js?5c40:6635)
at Proxy.render (Home.vue?bb51:2)
at renderComponentRoot (runtime-core.esm-bundler.js?5c40:1166)
at componentEffect (runtime-core.esm-bundler.js?5c40:5265)......
my product like :
[
{
"id": 1,
"name": "chair",
"categoryId": 12,
"unitId": 2,
"price": 66000000,
"salePrice": 0,
"material": "wood",
"size": "x"
},
]
My code here:
Home.vue file
<ProductCard v-for="product in products" :key="product.id" :product="product" />
ProductCard.vue file
<script>
export default {
name: "ProductCard",
props: {
product: {
type: Object,
required: true,
},
},
};
</script>
ProductService.js file
const apiClient = axios.create({
baseURL: 'http://localhost:8888/api/v1',
withCredentials: false,
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
},
})
export default {
getProducts() {
return apiClient.get('/product/get-list-product-by-subcategory')
},
}
When I print out list product in console. It still work.
Does anyone know where is the bug in my code?
Updated:
I try to fix my bug "Cannot read property 'id' of null", Steve's answer although remove my red warning in devtool but not deal my data: my data still not showing up. And I find out my code work by using this.products = response.data.data
ProductService.getProducts()
.then((response) => (this.products = response.data.data))
.catch((error) => console.log("error: " + error));
Explain by myself is:
When console.log(this.products = response)
And I need to use this.products = response.data.data to enter to array
apiClient.get(...)
returns a promise not the actual data from the API call.
You need to add a then. like so
apiClient.get(...).then(response => (this.products = response))
Then when the apiClient.get completes this.products will be populated with the data from the API.
Try this
<ProductCard v-for="product in products" :key="product._id" :product="product" />

Problem sending POST body to the Firestore REST API

I want to create a new document in Firestore using the REST API.
Very good examples here using Axios to send the POST request with some fields:
https://www.jeansnyman.com/posts/google-firestore-rest-api-examples/
axios.post(
"https://firestore.googleapis.com/v1/projects/<PROJECTIDHERE>/databases/(default)/documents/<COLLECTIONNAME>",
{
fields: {
title: { stringValue: this.title },
category: { stringValue: this.category },
post: { stringValue: this.post },
summary: { stringValue: this.description },
published: { booleanValue: this.published },
created: { timestampValue: new Date() },
modified: { timestampValue: new Date() }
}
}
).then(res => { console.log("Post created") })
And an example here using Python Requests:
Using the Firestore REST API to Update a Document Field
(this is a PATCH request but the field formatting is the same as in a POST request)
import requests
import json
endpoint = "https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default)/documents/[COLLECTION]/[DOCUMENT_ID]?currentDocument.exists=true&updateMask.fieldPaths=[FIELD_1]"
body = {
"fields" : {
"[FIELD_1]" : {
"stringValue" : "random new value"
}
}
}
data = json.dumps(body)
headers = {"Authorization": "Bearer [AUTH_TOKEN]"}
print(requests.patch(endpoint, data=data, headers=headers).json())
I am using Google Apps Script UrlFetchApp.fetch to send my requests. I am able to use GET requests with no problems. For example, to get all the documents in a collection (in Google Apps Script):
function firestore_get_documents(){
var options = {
headers: { 'Authorization': 'Bearer ' + ScriptApp.getOAuthToken() },
method:'GET'
}
var response = UrlFetchApp.fetch('https://firestore.googleapis.com/v1/projects/<PROJECTIDHERE>/databases/(default)/documents/myCollection', options);
var parsed = JSON.parse(response.getContentText());
return parsed;
}
This works nicely. And changing 'method' to 'POST' creates a new document in myCollection as expected. Then I try to add a POST body with some fields (or just one field):
function firestore_create_new_document(){
var options = {
headers: { 'Authorization': 'Bearer ' + ScriptApp.getOAuthToken() },
method:'POST',
payload: {fields: { title: { stringValue: 'newTitle' } } }, // If you comment out this line, it works as expected
muteHttpExceptions:true
}
var response = UrlFetchApp.fetch('https://firestore.googleapis.com/v1/projects/<PROJECTIDHERE>/databases/(default)/documents/myCollection', options);
var contentText = response.getContentText();
var parsed = JSON.parse(response.getContentText());
return parsed;
}
I get the following errors:
code: 400 message: "Request contains an invalid argument."
status: "INVALID_ARGUMENT"
details[0][#type]: "type.googleapis.com/google.rpc.BadRequest"
details[0][fieldViolations][0][field]: "{title={stringValue=newTitle}}"
details[0][fieldViolations][0][description]: "Error expanding 'fields' parameter. Cannot find matching fields for path '{title={stringValue=newTitle}}'."
Documentation is available here:
https://firebase.google.com/docs/firestore/reference/rest/v1/projects.databases.documents/createDocument
https://firebase.google.com/docs/firestore/reference/rest/v1/projects.databases.documents#Document
The problem may be the formatting of my 'fields' object - I've tried several different formats from the documentation and examples
The problem may be that the fields don't exist yet? I think I should be able to create a new document with new fields
The problem may be with the way UrlFetchApp.fetch sends my JSON body. I have tried using payload = JSON.stringify(payload_object) and that doesn't work either.
I think UrlFetchApp is doing something slightly different than Axios or Python Requests - the body is getting sent differently, and not parsing as expected.
How about the following modification?
From:
var options = {
headers: { 'Authorization': 'Bearer ' + ScriptApp.getOAuthToken() },
method:'POST',
payload: {fields: { title: { stringValue: 'newTitle' } } }, // If you comment out this line, it works as expected
muteHttpExceptions:true
}
To:
var options = {
headers: { 'Authorization': 'Bearer ' + ScriptApp.getOAuthToken() },
method:'POST',
payload: JSON.stringify({fields: { title: { stringValue: 'newTitle' } } }),
contentType: "application/json",
muteHttpExceptions:true
}
When I tested above modified request, I could confirm that it worked. But if other error occurs, please tell me.
Reference:
Class UrlFetchApp

Google chart using JSON - Format issue - ASP.net - "Table has no columns”

I am seeing 2 types of JSON format out there and getting really confused.
I a trying to do the following:
I have an excel model with a range
Type RV
MRC 10
CRC 20
CVA 30
OpRisk 0
Using ExCelToWeb vba function from cDataSet (http://ramblings.mcpher.com/Home/excelquirks/json/excel-json-conversion), I convert this range to a JSON string. Output is as follows:
{ "cDataSet":[
{
"Type":"MRC",
"RV":10
},
{
"Type":"CRC",
"RV":20
},
{
"Type":"CVA",
"RV":30
},
{
"Type":"OpRisk",
"RV":0
}
]}
I save this into a text file called myData.json
In Default.aspx.cs, I have
[WebMethod]
public static String GetDataJSON(){
String myvar = File.ReadAllText("C:\\Users\\Serge\\Downloads\\GoogleChartExample\\GoogleChartExample\\myData.json");
return myvar;
}
I call the function from Java Script in Default.aspx
google.load('visualization', '1', { packages: ['corechart'] });
google.setOnLoadCallback(drawChart);
function drawChart() {
$.ajax({
type: 'POST',
dataType: 'json',
url: 'Default.aspx/GetDataJSON',
contentType: 'application/json',
data: {},
async: false,
success:
function (response) {
console.log(response.d);
var data = new google.visualization.DataTable(response.d);
var csv = google.visualization.dataTableToCsv(data);
console.log(csv);
new google.visualization.ColumnChart(document.getElementById('myChart1')).draw(data, { title: "MyChart1" });
},
error: function (jqXHR, textStatus, errorThrown) {
alert(textStatus + '\n' + errorThrown);
if (!$.browser.msie) {
console.log(jqXHR);
}
}
});
}
=> console.log(response.d) outputs my JSON string exactly as intended
=> console.log(csv) does not output anything
=> the google chart outputs a red box saying “Table has no columns”
I understand that there are 2 types of JSON format. The one that google has in its example has "cols": [ , and "rows": [ . Does it mean that Google Chart / Dataset is not compatible with my format? Is there anyway for me to convert from one to the other?
Would I be better not to use JSON and export from excel -> cvs => a C# array -> a google dataset?
Thank you very much.
Serge

Resources