i could not able to parse the below json value , I tried with parse_json() and todynamic() ,I m getting the result column values to be empty
]1
the issue is that your payload includes an internal invalid JSON payload.
it is possible to "fix" it using the query language (see usages of replace() in the example below), however it'd be best if you can write a valid JSON payload to begin with.
try running this:
print s = #'{"pipelineId":"63dfc1f6-5a43-5bca-bffe-6a36a435e19d","vmId":"9252382a-814f-4d02-9b1b-305db4caa208/usl-exepipe-dev/westus/usl-exepipe-lab-dev/asuvp306563","artifactResult":{"Id":"execution-job-2","SourceName":"USL Repository","ArtifactName":"install-lcu","Status":"Succeeded","Parameters":null,"Log":"[{\"code\":\"ComponentStatus/StdOut/succeeded\",\"level\":\"Info\",\"displayStatus\":\"Provisioning succeeded\",\"message\":\"2020-06-02T14:33:04.711Z | I | Starting artifact ''install-lcu''\r\n2020-06-02T14:33:04.867Z | I | Starting Installation\r\n2020-06-02T14:33:04.899Z | I | C:\\USL\\LCU\\4556803.msu Exists.\r\n2020-06-02T14:33:04.914Z | I | Starting installation process ''C:\\USL\\LCU\\4556803.msu /quiet /norestart''\r\n2020-06-02T14:43:14.169Z | I | Process completed with exit code ''3010''\r\n2020-06-02T14:43:14.200Z | I | Need to restart computer after hotfix 4556803 installation\r\n2020-06-02T14:43:14.200Z | I | Finished Installation\r\n2020-06-02T14:43:14.200Z | I | Artifact ''install-lcu'' succeeded\r\n\",\"time\":null},{\"code\":\"ComponentStatus/StdErr/succeeded\",\"level\":\"Info\",\"displayStatus\":\"Provisioning succeeded\",\"message\":\"\",\"time\":null}]","DeploymentLog":null,"StartTime":"2020-06-02T14:32:40.9882134Z","ExecutionTime":"00:11:21.2468597","BSODCount":0},"attempt":1,"instanceId":"a301aaa0c2394e76832867bfeec04b5d:0","parentInstanceId":"78d0b036a5c548ecaafc5e47dcc76ee4:2","eventName":"Artifact Result"}'
| mv-expand log = parse_json(replace("\r\n", " ", replace(#"\\", #"\\\\", tostring(parse_json(tostring(parse_json(s).artifactResult)).Log))))
| project log.code, log.level, log.displayStatus, log.message
Related
I am getting error while executing OpenStack simple mistral workflow on OpenStack(wallaby) devstack environment. While I can execute the workflow from CLI command and got success But it fails if I try the same thing with GUI
root#openstack:~# openstack workflow definition show test_get
---
version: '2.0'
test_get:
description: Test Get.
tasks:
my_task:
action: std.http
input:
url: http://www.google.com
root#openstack:~# openstack workflow execution create test_get
+--------------------+--------------------------------------+
| Field | Value |
+--------------------+--------------------------------------+
| ID | 482e3803-45ef-411e-a0f4-1427abfc8649 |
| Workflow ID | 9dc0d4a4-8c5b-4288-8126-e1147da3bd02 |
| Workflow name | test_get |
| Workflow namespace | |
| Description | |
| Task Execution ID | <none> |
| Root Execution ID | <none> |
| State | RUNNING |
| State info | None |
| Created at | 2021-06-21 16:58:54 |
| Updated at | 2021-06-21 16:58:54 |
| Duration | ... |
+--------------------+--------------------------------------+
But while executing in GUI I get **
Execution is missing field "workflow_identifier"
**
Faced the same issue in Yoga release. Spent a few hours to investigate it and found interesting thing:
/usr/local/lib/python3.8/dist-packages/mistralclient/api/v2/executions.py
class ExecutionManager(base.ResourceManager):
resource_class = Execution
def create(self, wf_identifier='', namespace='',
workflow_input=None, description='', source_execution_id=None,
**params):
self._ensure_not_empty(
workflow_identifier=wf_identifier or source_execution_id
)
But! in the webform we are using workflow_identifier instead of wf_identifier
/usr/local/lib/python3.8/dist-packages/mistraldashboard/workflows/forms.py
def handle(self, request, data):
try:
data['workflow_identifier'] = data.pop('workflow_name')
data['workflow_input'] = {}
for param in self.workflow_parameters:
value = data.pop(param)
if value == "":
value = None
data['workflow_input'][param] = value
ex = api.execution_create(request, **data)
FIX is to rename workflow_identifier to wf_identifier in the form like
data['wf_identifier'] = data.pop('workflow_name')
After that mistral-dashboard works fine with execution creating.
I am writing firebase security rules, and I am attempting to get a document that may have a space in its documentID.
I have the following snippet which works well when the document does not have a space
function isAdminOfCompany(companyName) {
let company = get(/databases/$(database)/documents/Companies/$(companyName));
return company.data.authorizedUsers[request.auth.uid].access == "ADMIN;
}
Under the collection, "Companies", I have a document called "Test" and another called "Test Company" - Trying to get the document corresponding to "Test" works just fine, but "Test Company" does not seem to work, as the company variable (first line into the function) is equal to null as per the firebase security rules "playground".
My thought is that there is something to do with URL encoding, but replacing the space in a documentID to "%20" or a "+" does not change the result. Perhaps spaces are illegal characters for documentIDs (https://cloud.google.com/firestore/docs/best-practices lists a few best practices)
Any help would be appreciated!
EDIT: As per a few comments, I Will add some additional images/explanations below.
Here is the structure of my database
And here is what fields are present in the user documents
In short, the following snippet reproduces the problem (I am not actually using this, but it demonstrates the issue the same way)
rules_version = '2';
service cloud.firestore {
match /databases/{database}/documents {
match /Users/{user} {
allow update: if findPermission(resource.data.company) == "MASTER"
}
function findPermission(companyName) {
let c = get(path("/databases/" + database + "/documents/Companies/" + companyName));
return c.data.authorizedUsers[request.auth.uid].access;
}
}
}
When I try to update a user called test#email.com (which belongs to company "Test"), the operation is permitted, and everything works exactly as expected.
The issue arises when a user, called test2#email.com, who belongs to company "Test Company" comes along and makes the same request (with authorization email/uid updated in playground, to match what is actually found in the company structure), the request fails. The request fails, since the get() call (line 1 of the function) cannot find the Company document corresponding to "Test Company" - indicated by the variable "c" being null in the screenshot (see below) - IT IS NOT NULL WHEN LOOKING FOR "Test"
Below is a screenshot of the error message, as well as some of the relevant variables when the error occurs
Check to see what type of space just in case it is another non-printable character. You could convert it to Unicode, and check what it might be. However, it is considered bad practice to use spaces in naming variables and data structures. There are so many different types to consider.
| Unicode | HTML | Description | Example |
|---------|--------|--------------------|---------|
| U+0020 |   | Space | [ ] |
| U+00A0 |   | No-Break Space | [ ] |
| U+2000 |   | En Quad | [ ] |
| U+2001 |   | Em Quad | [ ] |
| U+2002 |   | En Space | [ ] |
| U+2003 |   | Em Space | [ ] |
| U+2004 |   | Three-Per-Em Space | [ ] |
| U+2005 |   | Four-Per-Em Space | [ ] |
| U+2006 |   | Six-Per-Em Space | [ ] |
| U+2007 |   | Figure Space | [ ] |
| U+2008 |   | Punctuation Space | [ ] |
| U+2009 |   | Thin Space | [ ] |
| U+200A |   | Hair Space | [ ] |
On my method accountRepo.deleteAll()
return error:
2020-06-04 15:37:05.069 ERROR 78168 --- [ main] o.h.engine.jdbc.spi.SqlExceptionHelper : ОШИБКА: UPDATE or DELETE in table "account" violates foreign key constraint "fk8k31xl4ld2m810mxfkqp2xg8g" from table "tokens"
Details: on a key (account_id)=(2) there are still links in the table "tokens".
2020-06-04 15:37:05.071 INFO 78168 --- [ main] o.h.e.j.b.internal.AbstractBatchImpl : HHH000010: On release of batch it still contained JDBC statements
org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint [null]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
What could be the problem? This happens when creating a post-request with the registration of a user who will fall into accountRepo
My test and post-request:
#Test
#Throws(Exception::class)
fun shouldRegistrationExpected201() {
val headers = HttpHeaders()
headers.contentType = MediaType.APPLICATION_JSON
val request = HttpEntity<String>("{\"username\": \"holker228\", \"password\": \"123456QQwerty&&\",\"email\":\"test2#mail.ru\",\"is_collective\": \"false\" }", headers)
val responseEntity = restTemplate.postForEntity("http://localhost:$port/api/user/registration", request, String::class.java)
assertEquals(responseEntity.statusCode, HttpStatus.CREATED)
}
The issue seems to be in constrains configured in your database. BTW, which one do you use? In your particular case the constraint disallows DELETE (UPDATE?) operation as the row you're trying to delete seems to have dependent rows in another table.
E.g, imagine you have a table "users" and "comments":
users:
-------------
| ID | Name |
-------------
| 1 | Max |
-------------
| 2 | Vova |
-------------
| 3 | Vlad |
-------------
comments:
---------------------------------
| ID | User ID | Text |
---------------------------------
| 1 | 1 | Max was here! |
---------------------------------
| 2 | 1 | Vova + Vlad = ❤️ |
---------------------------------
| 3 | 2 | Don't listen to Max! |
---------------------------------
Here, if the constrains are configured, you cannot delete Max (1) or Vova (2) from "users" as they have comments. Imagine, you're rendering an HTML page and you need to render a comment's author's name? What would you do, if the author is deleted? (well, you can use "Anonymous", but let's not think about it). You cannot either change Max's or Vova's IDs for the same reason. So that's what constraints are for: they prevent such unconsistencies.
Ways to deal with this situation:
Update the related entities in the same transaction.
Use CASCADE for DELETEs. Take a look at this answer, for example
Freeze the IDs and do not change them.
How do I determine what the SQL statement is of the thread ID showing up in a metadata lock info row (SELECT * FROM information_schema.metadata_lock_info) on MariaDB?
Server version: 10.0.15-MariaDB MariaDB Server
All of the related questions dive into the "Waiting for table metadata lock" from a MySQL perspective, but that does not help with MariaDB since their introspection is implemented differently from what I can tell. Googling around does not turn up a whole lot.
A "show full processlist" gives rows like:
| 57295 | main | localhost | joints | Execute | 50 | Waiting for table metadata lock | select ...
Which does show the statement, but does not show that it has the lock either. So, I turned on metadata lock info as explained here [0]. This only provides the thread ID of the lock holder, but not the statement:
MariaDB [joints]> SELECT * FROM information_schema.metadata_lock_info;
+-----------+--------------------------+-----------------+----------------------+--------------+----------------+
| THREAD_ID | LOCK_MODE | LOCK_DURATION | LOCK_TYPE | TABLE_SCHEMA | TABLE_NAME |
+-----------+--------------------------+-----------------+----------------------+--------------+----------------+
| 57322 | MDL_INTENTION_EXCLUSIVE | MDL_EXPLICIT | Global read lock | | |
| 57322 | MDL_SHARED_NO_READ_WRITE | MDL_EXPLICIT | Table metadata lock | joints | 16_study |
| 57322 | MDL_INTENTION_EXCLUSIVE | MDL_EXPLICIT | Schema metadata lock | joints | |
| 57269 | MDL_SHARED_READ | MDL_TRANSACTION | Table metadata lock | joints | authentication |
| 57301 | MDL_SHARED_READ | MDL_TRANSACTION | Table metadata lock | joints | authentication |
| 57280 | MDL_SHARED_READ | MDL_TRANSACTION | Table metadata lock | joints | authentication |
| 57317 | MDL_SHARED_READ | MDL_TRANSACTION | Table metadata lock | joints | ship |
| 57271 | MDL_SHARED_READ | MDL_TRANSACTION | Table metadata lock | joints | administration |
| 57264 | MDL_SHARED_READ | MDL_TRANSACTION | Table metadata lock | joints | server |
+-----------+--------------------------+-----------------+----------------------+--------------+----------------+
What I really want is to see the "join" of both of those outputs at the moment the locking is happening. I do not see a way to join the data from these two "tables" since the former does not appear to be a table. I'd like to avoid getting:
ERROR 1933 (HY000): Target is not running an EXPLAINable command
while attempting to do it in real-time, due to the thread ending while being inspected.
[0] https://mariadb.com/kb/en/mariadb/metadata_lock_info/
THREAD_ID maps to information_schema.PROCESSLIST.ID (the first column in show
[full] processlist;. ie:
SELECT * FROM information_schema.METADATA_LOCK_INFO AS mli
JOIN information_schema.PROCESSLIST AS pl ON mli.THREAD_ID = pl.ID
I am preferential towards something like the following to make it easier to see the what is happening (the newlines don't work well with the cli though):
SELECT
mli.THREAD_ID, mli.LOCK_MODE, mli.LOCK_TYPE,
CAST(GROUP_CONCAT(DISTINCT CONCAT(mli.TABLE_SCHEMA, '.', mli.TABLE_NAME) ORDER BY mli.TABLE_SCHEMA, mli.TABLE_NAME SEPARATOR '\n') AS CHAR) AS locked_tables,
pl.USER, pl.HOST, pl.DB, pl.COMMAND, pl.TIME, pl.STATE, pl.INFO, pl.QUERY_ID, pl.TID
FROM information_schema.METADATA_LOCK_INFO AS mli
JOIN information_schema.PROCESSLIST AS pl ON mli.THREAD_ID = pl.ID
GROUP BY mli.THREAD_ID, mli.LOCK_MODE, mli.LOCK_TYPE
ORDER BY time DESC, pl.ID;
Especially interesting is when pl.COMMAND = 'Sleep' as that indicates some connection pool or other (mostly read-only) program is holding open connections that have locks on them.
I coded a java client that sends a string of meta information and a byte array through a multipart post http request to my server running cherrypy 3.6.
I need to extract both values and I coded this in python3 on the server side to find out how to manipulate the result as I can't find any relevant documentation over internet that explains how to read this html part
def controller(self, meta, data):
print("meta", meta)
print("data", type(data))
outputs :
my meta information
<class 'cherrypy._cpreqbody.Part'>
Note : the data part contains raw binary data.
How can I read the http part content into a buffer or output it to a disk file ?
Thanks for your help.
Thanks for your answer.
I'v already read this doc but unfortunately methods read-into_file and make_file, read ... it doesn't work for me. for example when trying to read a zip file sent form my java client :
Assuming data is the Http post parameter
make_file()
fp = data.make_file()
print("fp type", type(fp)) # _io.BufferedRandom
zipFile = fp.read()
outputs:
AttributeError: 'bytes' object has no attribute 'seek'
line 651, in read_lines_to_boundary raise EOFError("Illegal end of multipart body.")EOFError: Illegal end of multipart body.
read_into_file()
file = data.read_into_file()
print("file type", type(file))
zipFile = io.BytesIO(file.read())
# zipFile = file.read() # => raises same error
outputs:
line 651, in read_lines_to_boundary raise EOFError("Illegal end of multipart body.")EOFError: Illegal end of multipart body.
I don't understand what happens ...
Actually "data" is not a file like object but a cherrypy._cpreqbody.Part one. It holds a "file" file an _io.BufferedRandom class property.
Its read() method returns the whole body content in a binary form (bytes).
so to end up the straightforward solution is :
class BinReceiver(object):
def index(self, data):
zipFile = io.BytesIO(data.file.read())
path = "/tmp/data.zip"
fp = open(path)
fp.write(zipFile, 'wb')
fp.close()
print("saved data into", path, "size", len(zipFile))
index.exposed = True
and this works fine ...
fyi : I'm running python3.2
It seems like data is a file-like object which you can call .read on. In addition CherryPy provides a method read_into_file.
See the full documentation by typing help(cherrypy._cpreqbody.Part) in your REPL.
class Part(Entity)
| A MIME part entity, part of a multipart entity.
|
| Method resolution order:
| Part
| Entity
| __builtin__.object
|
| Methods defined here:
|
| __init__(self, fp, headers, boundary)
|
| default_proc(self)
| Called if a more-specific processor is not found for the
| ``Content-Type``.
|
| read_into_file(self, fp_out=None)
| Read the request body into fp_out (or make_file() if None).
|
| Return fp_out.
|
| read_lines_to_boundary(self, fp_out=None)
| Read bytes from self.fp and return or write them to a file.
|
| If the 'fp_out' argument is None (the default), all bytes read are
| returned in a single byte string.
|
| If the 'fp_out' argument is not None, it must be a file-like
| object that supports the 'write' method; all bytes read will be
| written to the fp, and that fp is returned.
|
| ----------------------------------------------------------------------
| Class methods defined here:
|
| from_fp(cls, fp, boundary) from __builtin__.type
|
| read_headers(cls, fp) from __builtin__.type
|
| ----------------------------------------------------------------------
| Data and other attributes defined here:
|
| attempt_charsets = ['us-ascii', 'utf-8']
|
| boundary = None
|
| default_content_type = 'text/plain'
|
| maxrambytes = 1000
|
| ----------------------------------------------------------------------
| Methods inherited from Entity:
|
| __iter__(self)
|
| __next__(self)
|
| fullvalue(self)
| Return this entity as a string, whether stored in a file or not.
|
| make_file(self)
| Return a file-like object into which the request body will be read.
|
| By default, this will return a TemporaryFile. Override as needed.
| See also :attr:`cherrypy._cpreqbody.Part.maxrambytes`.
|
| next(self)
|
| process(self)
| Execute the best-match processor for the given media type.
|
| read(self, size=None, fp_out=None)
|
| readline(self, size=None)
|
| readlines(self, sizehint=None)
|
| ----------------------------------------------------------------------
| Data descriptors inherited from Entity:
|
| __dict__
| dictionary for instance variables (if defined)
|
| __weakref__
| list of weak references to the object (if defined)
|
| type
| A deprecated alias for :attr:`content_type<cherrypy._cpreqbody.Entity.content_type>`.
|
| ----------------------------------------------------------------------
| Data and other attributes inherited from Entity:
|
| charset = None
|
| content_type = None
|
| filename = None
|
| fp = None
|
| headers = None
|
| length = None
|
| name = None
|
| params = None
|
| part_class = <class 'cherrypy._cpreqbody.Part'>
| A MIME part entity, part of a multipart entity.
|
| parts = None
|
| processors = {'application/x-www-form-urlencoded': <function process_u...