I encounter a problem in my production site in Symfony 4.2.8 (yes I know...).
When I try to launch a bin/console fos:elastica:populate or bin/console fos:elastica:reset the console crash (with -vvv option) :
Exception trace:
() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Transport/Http.php:190
Elastica\Transport\Http->exec() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Request.php:194
Elastica\Request->send() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Client.php:689
Elastica\Client->request() at /var/www/current/vendor/friendsofsymfony/elastica-bundle/src/Elastica/Client.php:58
FOS\ElasticaBundle\Elastica\Client->request() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Client.php:721
Elastica\Client->requestEndpoint() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Index.php:586
Elastica\Index->requestEndpoint() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Index.php:225
Elastica\Index->delete() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Index.php:296
Elastica\Index->create() at /var/www/current/vendor/friendsofsymfony/elastica-bundle/src/Index/Resetter.php:110
FOS\ElasticaBundle\Index\Resetter->resetIndex() at /var/www/current/vendor/friendsofsymfony/elastica-bundle/src/Command/PopulateCommand.php:184
FOS\ElasticaBundle\Command\PopulateCommand->populateIndex() at /var/www/current/vendor/friendsofsymfony/elastica-bundle/src/Command/PopulateCommand.php:164
FOS\ElasticaBundle\Command\PopulateCommand->execute() at /var/www/current/vendor/symfony/console/Command/Command.php:255
Symfony\Component\Console\Command\Command->run() at /var/www/current/vendor/symfony/console/Application.php:926
Symfony\Component\Console\Application->doRunCommand() at /var/www/current/vendor/symfony/framework-bundle/Console/Application.php:89
Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand() at /var/www/current/vendor/symfony/console/Application.php:269
Symfony\Component\Console\Application->doRun() at /var/www/current/vendor/symfony/framework-bundle/Console/Application.php:75
Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /var/www/current/vendor/symfony/console/Application.php:145
Symfony\Component\Console\Application->run() at /var/www/current/bin/console:39
In Http.php line 190:
[Elastica\Exception\Connection\HttpException]
Couldn't connect to host, Elasticsearch down?
A curl on ES server respond :
{
"name" : "prod",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "********",
"version" : {
"number" : "7.10.0",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "51e9d6f22758d0374a0f3f5c6e8f3a7997850f96",
"build_date" : "2020-11-09T21:30:33.964949Z",
"build_snapshot" : false,
"lucene_version" : "8.7.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
I create a new branch to upgrade all the component but I need a clue to operate a quick fix on production...
If somebody have an idea thank in advance.
Related
I'm trying to integrate Kibana with my OpenSearch ( is it possible ? ). Unfortunately I get a version error.
Is there any way to use opensearch in kibana?
These are the versions I get via curl:
curl -k -u "admin:PASSWORD" "https://IP:9200/"
{
"name" : "node-1",
"cluster_name" : "cluster",
"cluster_uuid" : "he6gqhl2S-6dlVv6dyPOEA",
"version" : {
"number" : "7.10.2",
"build_type" : "rpm",
"build_hash" : "e505b10357c03ae8d26d675172402f2f2144ef0f",
"build_date" : "2022-01-14T03:38:06.881862Z",
"build_snapshot" : false,
"lucene_version" : "8.10.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "The OpenSearch Project: https://opensearch.org/"
}
They are very tightly coupled and aren't really supposed to work together, so I doubt they will (or it would be very brittle).
If you want to have Kibana, why not Elasticsearch under the hood?
I am trying to fetch the SHA_256 value from manifest.json file but unable to get using aql.
Below is the cmd I am using:
ubuntu#test:~$ **curl -sS -u sumkumar:$pw -XPOST -k -H "Content-type: text/plain" https://<URL>/artifactory/api/search/aql -d 'items.find({"repo":"xyz"},{"path":"a/b/c"}).include("*")'**
{
"results" : [ {
"repo" : "xyz",
"path" : "a/b/c",
"name" : "manifest.json",
"type" : "file",
"size" : 1579,
"created" : "2018-03-13T11:58:33.771Z",
"created_by" : "uex-sp-cd",
"modified" : "2018-03-15T14:17:38.299Z",
"modified_by" : "uex-sp-cd",
"updated" : "2018-03-15T14:17:38.299Z",
"depth" : 4,
"actual_md5" : "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"actual_sha1" : "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"original_md5" : "NO_ORIG",
"original_sha1" : "NO_ORIG",
"virtual_repos" : [ ]
},{
However, if you look the original manifest.json file from UI.It contains SHA256 value.
enter image description here
According to the AQL documentation, the item entity has the sha256 field. But it mentions the following note:
SHA-256 is supported from Artifactory version 5.5
You can only do an AQL search on an Artifact that has been deployed to Artifactory version 5.5 or above, or if you have migrated your database as described in SHA-256 Support after upgrading Artifactory to version 5.5 and above.
Please verify your Artifactory fits the requirements above.
In addition, assuming this is a Docker manifest.json, it specifically should also have the SHA256 in a property named docker.manifest.digest (and maybe also a property named sha256). To get the property values you can add "property.*" to the include(..) part of the query.
For example:
items.find({"repo":"xyz","path":"a/b/c","name":"manifest.json"})
.include("repo","path","name","sha256","property.*")
Will return something like:
{
"results" : [ {
"repo" : "xyz",
"path" : "a/b/c",
"name" : "manifest.json",
"sha256" : "34cb6f8e1e1aca...",
"properties" : [ {
"key" : "docker.manifest.digest",
"value" : "sha256:34cb6f8e1e1aca..."
},{
"key" : "sha256",
"value" : "34cb6f8e1e1aca..."
}
...
Objective
Refresh a partition using a query (override). Using these as a guide:
https://www.sqlbi.com/articles/using-process-add-in-tabular-models/
https://gist.github.com/dgosbell/a7bc9fe9ff5a99fdb4df5819b8760217#file-refresh-with-override-example-txt
Apparently the MS example is not correct: https://learn.microsoft.com/en-us/bi-reference/tmsl/refresh-command-tmsl#examples
TMSL Script
{
"refresh" : {
"type" : "add",
"objects" : [{
"database" : "dbname",
"table" : "tblname"
}
],
"overrides" : [{
"partitions" : [
{
"originalObject" : {
"database" : "dbname",
"table" : "tblname",
"partition" : "partname"
},
"source" : {
"query" :
"SELECT * FROM source.view WHERE date_field = '2014-12-06'"
}
}
]
}
]
}
}
Error Message
The JSON DDL request failed with the following error: Failed to execute XMLA. Error returned: 'The Process command for partition 'partname' in table 'tblname' cannot be executed because the pipeline rowset is missing.
'..
Technical Details:
RootActivityId: 89a6f9ac-e5d4-4eaa-b049-455190039b4b
Date (UTC): 6/28/2019 3:20:36 PM
0: PFError::SetLastError() line 2158 + 0x0 (sql\picasso\engine\src\pf\eh\pferror.cpp)
1: PFSetLastError() line 2906 + 0x0 (sql\picasso\engine\src\pf\eh\pferror.cpp)
2: ConvertExceptionsToPFResult<<lambda_764f81a97ea803a6bb1663c7971ce151> >() line 424 + 0x34 (sql\picasso\engine\src\pf\kernel\shared\pfshmacros.inl)
3: PFSetLastErrorExTag() line 3461 + 0x2e (sql\picasso\engine\src\pf\eh\pferror.cpp)
4: 0x00007FFAB599CC7E (symbolic name unavailable)
Other Info
Executed on SSMS directly and in Powershell (via Runbook) with same error message.
Question
What does this error message mean exactly? (It is very hard to find
helpful documentation.) Or, is there an alternative solution to refreshing
a partition using a query override?
You can only use either a Query or M partition sources
If you are using an M partition source the syntax is:
"source":{
"type":"m",
"expression":"…"
}
And if you are using a Query partition source the syntax is:
"source":{
"type":"query",
"query":"…",
"dataSource":"…"
}
When I try to use httpoison to query an elasticsearch server like
iex(1)> HTTPoison.get "http://localhost:9200"
I get
{:error, %HTTPoison.Error{id: nil, reason: :econnrefused}}.
If I do
curl -XGET "http://localhost:9200"
I get
{
"name" : "es01",
"cluster_name" : "docker-cluster",
"cluster_uuid" : "Wik-EpMkQ8ummJE6ctNAOg",
"version" : {
"number" : "7.0.1",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "e4efcb5",
"build_date" : "2019-04-29T12:56:03.145736Z",
"build_snapshot" : false,
"lucene_version" : "8.0.0",
"minimum_wire_compatibility_version" : "6.7.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
Does anyone know what this behavior is due to and how to fix it?
P.S.: Changing localhost to 127.0.0.1 does not solve the problem.
Here's my setup:
elasticsearch Version: 7.0.1
{:httpoison, "~> 1.5"} #=> mix.lock shows version 1.5.1 was installed
curl results:
$ curl -XGET "http://localhost:9200"
{
"name" : "My-MacBook-Pro.local",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "vEFl3B5TTYaBxPhQFuXPyQ",
"version" : {
"number" : "7.0.1",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "e4efcb5",
"build_date" : "2019-04-29T12:56:03.145736Z",
"build_snapshot" : false,
"lucene_version" : "8.0.0",
"minimum_wire_compatibility_version" : "6.7.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
HTTPoison results:
$ iex -S mix
Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]
===> Compiling parse_trans
===> Compiling mimerl
===> Compiling metrics
===> Compiling unicode_util_compat
===> Compiling idna
==> ssl_verify_fun
Compiling 7 files (.erl)
Generated ssl_verify_fun app
===> Compiling certifi
===> Compiling hackney
==> httpoison
Compiling 3 files (.ex)
Generated httpoison app
==> hello
Compiling 15 files (.ex)
Generated hello app
Interactive Elixir (1.6.6) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> HTTPoison.get "http://localhost:9200"
{:ok,
%HTTPoison.Response{
body: "{\n \"name\" : \"My-MacBook-Pro.local\",\n \"cluster_name\" :
\"elasticsearch\",\n \"cluster_uuid\" : \"vEFl3B5TTYaBxPhQFuXPyQ\",\n
\"version\" : {\n \"number\" : \"7.0.1\",\n \"build_flavor\" :
\"default\",\n \"build_type\" : \"tar\",\n \"build_hash\" :
\"e4efcb5\",\n \"build_date\" : \"2019-04-29T12:56:03.145736Z\",\n
\"build_snapshot\" : false,\n \"lucene_version\" : \"8.0.0\",\n
\"minimum_wire_compatibility_version\" : \"6.7.0\",\n
\"minimum_index_compatibility_version\" : \"6.0.0-beta1\"\n },\n
\"tagline\" : \"You Know, for Search\"\n}\n",
headers: [
{"content-type", "application/json; charset=UTF-8"},
{"content-length", "522"}
],
request: %HTTPoison.Request{
body: "",
headers: [],
method: :get,
options: [],
params: %{},
url: "http://localhost:9200"
},
request_url: "http://localhost:9200",
status_code: 200
}}
iex(2)>
Next, I stopped the elasticsearch server, then I ran the HTTPoison request again:
ex(2)> HTTPoison.get "http://localhost:9200"
{:error, %HTTPoison.Error{id: nil, reason: :econnrefused}}
I got similar results for the curl request:
$ curl -XGET "http://localhost:9200"
curl: (7) Failed to connect to localhost port 9200: Connection refused
What happens if you issue two curl requests in a row? Do they both succeed? Try issuing the HTTPoison request first, then the curl request. Which one fails? Try the reverse order. Same results?
I'm almost positive you have the same problem that I did. Check to make sure you are not forcing ipv6 in your /etc/hosts file.
If you have something like this:
::1 localhost
...get rid of it and Httpoison should work again
I have build a parcel and a csd that go together and work fine when deploying to a cluster.
However, when I stop the service a child process started by a startup script keeps on running in the background.
I have tried many things but I went back to something more "brutal".
CSD extract
"startRunner" : {
"program" : "scripts/rexster.sh",
"args" : [ "start" ],
"environmentVariables" : {
"CONF_FILE" : "${conf_file}",
"REXSTER_PORT" : "${rexster_port}",
"HBASE_ZK" : "${hbase_zk_quorum}",
"REXSTER_SHUTDOWN_PORT" : "${rexster_shutdown_port}",
"HBASE_TABLE_NAME" : "${hbase_tablename}",
"MEM_XMX" : "${memory_xmx}",
"MEM_XMS" : "${memory_xms}"
}
},
"stopRunner" : {
"relevantRoleTypes" : ["TITAN_REXTER_SERVER"],
"runner" : {
"program" : "scripts/rexster.sh",
"args" : [ "stop" ]
}
}
}
scripts/rexster.sh extract
...
;;
(stop)
echo "Stopping rexster"
exec kill -9 `ps aux | grep java | grep titan | awk '{print $2}'`
# exec stopRexster.sh $TITAN_HOME
;;
(*)
echo "Don't understand [$CMD]"
;;
esac
But the process keeps on running in the background:
usr/java/jdk1.7.0_67-cloudera/bin/java -server -Xms128m -Xmx512m -Dtitan.logdir=../log com.tinkerpop.rexster.Application -s -c /opt/cloudera/parcels/TITAN-1.0/lib/titan-0.5.2-hadoop2-CDH5.3/conf/rexster-hbase.xml
The stoprunner wasn't placed correctly.
It is defined per role, after the role definition.
<...ROLE...>}]
,
"stopRunner" : {
"masterRole":"TITAN_REXTER_SERVER",
"relevantRoleTypes" : ["TITAN_REXTER_SERVER"],
"runner" : {
"program" : "scripts/rexster.sh",
"args" : [ "stop" ],
"environmentVariables" : {
"CONF_FILE" : "${conf_file}",
"REXSTER_PORT" : "${rexster_port}",
"HBASE_ZK" : "${hbase_zk_quorum}",
"REXSTER_SHUTDOWN_PORT" : "${rexster_shutdown_port}",
"HBASE_TABLE_NAME" : "${hbase_tablename}"
}
}
}