Httpoison request to elasticsearch server gives me econnrefused error, while curling the same thing doesn't - http

When I try to use httpoison to query an elasticsearch server like
iex(1)> HTTPoison.get "http://localhost:9200"
I get
{:error, %HTTPoison.Error{id: nil, reason: :econnrefused}}.
If I do
curl -XGET "http://localhost:9200"
I get
{
"name" : "es01",
"cluster_name" : "docker-cluster",
"cluster_uuid" : "Wik-EpMkQ8ummJE6ctNAOg",
"version" : {
"number" : "7.0.1",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "e4efcb5",
"build_date" : "2019-04-29T12:56:03.145736Z",
"build_snapshot" : false,
"lucene_version" : "8.0.0",
"minimum_wire_compatibility_version" : "6.7.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
Does anyone know what this behavior is due to and how to fix it?
P.S.: Changing localhost to 127.0.0.1 does not solve the problem.

Here's my setup:
elasticsearch Version: 7.0.1
{:httpoison, "~> 1.5"} #=> mix.lock shows version 1.5.1 was installed
curl results:
$ curl -XGET "http://localhost:9200"
{
"name" : "My-MacBook-Pro.local",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "vEFl3B5TTYaBxPhQFuXPyQ",
"version" : {
"number" : "7.0.1",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "e4efcb5",
"build_date" : "2019-04-29T12:56:03.145736Z",
"build_snapshot" : false,
"lucene_version" : "8.0.0",
"minimum_wire_compatibility_version" : "6.7.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
HTTPoison results:
$ iex -S mix
Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]
===> Compiling parse_trans
===> Compiling mimerl
===> Compiling metrics
===> Compiling unicode_util_compat
===> Compiling idna
==> ssl_verify_fun
Compiling 7 files (.erl)
Generated ssl_verify_fun app
===> Compiling certifi
===> Compiling hackney
==> httpoison
Compiling 3 files (.ex)
Generated httpoison app
==> hello
Compiling 15 files (.ex)
Generated hello app
Interactive Elixir (1.6.6) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> HTTPoison.get "http://localhost:9200"
{:ok,
%HTTPoison.Response{
body: "{\n \"name\" : \"My-MacBook-Pro.local\",\n \"cluster_name\" :
\"elasticsearch\",\n \"cluster_uuid\" : \"vEFl3B5TTYaBxPhQFuXPyQ\",\n
\"version\" : {\n \"number\" : \"7.0.1\",\n \"build_flavor\" :
\"default\",\n \"build_type\" : \"tar\",\n \"build_hash\" :
\"e4efcb5\",\n \"build_date\" : \"2019-04-29T12:56:03.145736Z\",\n
\"build_snapshot\" : false,\n \"lucene_version\" : \"8.0.0\",\n
\"minimum_wire_compatibility_version\" : \"6.7.0\",\n
\"minimum_index_compatibility_version\" : \"6.0.0-beta1\"\n },\n
\"tagline\" : \"You Know, for Search\"\n}\n",
headers: [
{"content-type", "application/json; charset=UTF-8"},
{"content-length", "522"}
],
request: %HTTPoison.Request{
body: "",
headers: [],
method: :get,
options: [],
params: %{},
url: "http://localhost:9200"
},
request_url: "http://localhost:9200",
status_code: 200
}}
iex(2)>
Next, I stopped the elasticsearch server, then I ran the HTTPoison request again:
ex(2)> HTTPoison.get "http://localhost:9200"
{:error, %HTTPoison.Error{id: nil, reason: :econnrefused}}
I got similar results for the curl request:
$ curl -XGET "http://localhost:9200"
curl: (7) Failed to connect to localhost port 9200: Connection refused
What happens if you issue two curl requests in a row? Do they both succeed? Try issuing the HTTPoison request first, then the curl request. Which one fails? Try the reverse order. Same results?

I'm almost positive you have the same problem that I did. Check to make sure you are not forcing ipv6 in your /etc/hosts file.
If you have something like this:
::1 localhost
...get rid of it and Httpoison should work again

Related

Kibana and OpenSearch incompatible versions

I'm trying to integrate Kibana with my OpenSearch ( is it possible ? ). Unfortunately I get a version error.
Is there any way to use opensearch in kibana?
These are the versions I get via curl:
curl -k -u "admin:PASSWORD" "https://IP:9200/"
{
"name" : "node-1",
"cluster_name" : "cluster",
"cluster_uuid" : "he6gqhl2S-6dlVv6dyPOEA",
"version" : {
"number" : "7.10.2",
"build_type" : "rpm",
"build_hash" : "e505b10357c03ae8d26d675172402f2f2144ef0f",
"build_date" : "2022-01-14T03:38:06.881862Z",
"build_snapshot" : false,
"lucene_version" : "8.10.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "The OpenSearch Project: https://opensearch.org/"
}
They are very tightly coupled and aren't really supposed to work together, so I doubt they will (or it would be very brittle).
If you want to have Kibana, why not Elasticsearch under the hood?

How to fetch SHA_256 value from manifest.json file in JFROG

I am trying to fetch the SHA_256 value from manifest.json file but unable to get using aql.
Below is the cmd I am using:
ubuntu#test:~$ **curl -sS -u sumkumar:$pw -XPOST -k -H "Content-type: text/plain" https://<URL>/artifactory/api/search/aql -d 'items.find({"repo":"xyz"},{"path":"a/b/c"}).include("*")'**
{
"results" : [ {
"repo" : "xyz",
"path" : "a/b/c",
"name" : "manifest.json",
"type" : "file",
"size" : 1579,
"created" : "2018-03-13T11:58:33.771Z",
"created_by" : "uex-sp-cd",
"modified" : "2018-03-15T14:17:38.299Z",
"modified_by" : "uex-sp-cd",
"updated" : "2018-03-15T14:17:38.299Z",
"depth" : 4,
"actual_md5" : "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"actual_sha1" : "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"original_md5" : "NO_ORIG",
"original_sha1" : "NO_ORIG",
"virtual_repos" : [ ]
},{
However, if you look the original manifest.json file from UI.It contains SHA256 value.
enter image description here
According to the AQL documentation, the item entity has the sha256 field. But it mentions the following note:
SHA-256 is supported from Artifactory version 5.5
You can only do an AQL search on an Artifact that has been deployed to Artifactory version 5.5 or above, or if you have migrated your database as described in SHA-256 Support after upgrading Artifactory to version 5.5 and above.
Please verify your Artifactory fits the requirements above.
In addition, assuming this is a Docker manifest.json, it specifically should also have the SHA256 in a property named docker.manifest.digest (and maybe also a property named sha256). To get the property values you can add "property.*" to the include(..) part of the query.
For example:
items.find({"repo":"xyz","path":"a/b/c","name":"manifest.json"})
.include("repo","path","name","sha256","property.*")
Will return something like:
{
"results" : [ {
"repo" : "xyz",
"path" : "a/b/c",
"name" : "manifest.json",
"sha256" : "34cb6f8e1e1aca...",
"properties" : [ {
"key" : "docker.manifest.digest",
"value" : "sha256:34cb6f8e1e1aca..."
},{
"key" : "sha256",
"value" : "34cb6f8e1e1aca..."
}
...

Unable to verify the first certificate - kibana/elasticsearch

I'm trying to configure xpack for elasticsearch/kibana, I've activated the trial license for elasticsearch, configured xpack for kibana/elasticsearch and also I've generated ca.crt, node1-elk.crt, node1-elk.key and also kibana.key , kibana.crt and if I'm testing with curl towards the elasticsearch using the kibana user and password and also the ca.crt it's working like a charm, if I'm trying to access kibana from the GUI says that the "Server is not ready yet" and the logs show that " unable to verify the first certificate" :
{"type":"log","#timestamp":"2021-11-16T04:41:09-05:00","tags":["error","savedobjects-service"],"pid":13250,"message":"Unable to retrieve version information from Elasticsearch nodes. unable to verify the first certificate"}
My configuration:
kibana.yml
server.name: "my-kibana"
server.host: "0.0.0.0"
elasticsearch.hosts: ["https://0.0.0.0:9200"]
server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/certs/kibana.crt
server.ssl.key: /etc/kibana/certs/kibana.key
server.ssl.certificateAuthorities: ["/etc/kibana/certs/ca.crt"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "kibana"
elasticsearch.yml
node.name: node1
network.host: 0.0.0.0
discovery.seed_hosts: [ "0.0.0.0" ]
cluster.initial_master_nodes: ["node1"]
xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.key: /etc/elasticsearch/certs/node1.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/node1.crt
xpack.security.http.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
xpack.security.transport.ssl.key: /etc/elasticsearch/certs/node1.key
xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/node1.crt
xpack.security.transport.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
curl testing:
[root#localhost kibana]# curl -XGET https://0.0.0.0:9200/_cat/nodes?v -u kibana_system:kibana --cacert /etc/elasticsearch/certs/ca.crt
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.100.102 23 97 3 0.00 0.02 0.08 cdfhilmrstw * node1
I don't know what to do more here:
[root#localhost kibana]# curl -XGET https://0.0.0.0:9200/_license -u kibana_system:kibana --cacert /etc/elasticsearch/certs/ca.crt
{
"license" : {
"status" : "active",
"uid" : "872f0ad0-723e-43c8-b346-f43e2707d3de",
"type" : "trial",
"issue_date" : "2021-11-08T18:26:15.422Z",
"issue_date_in_millis" : 1636395975422,
"expiry_date" : "2021-12-08T18:26:15.422Z",
"expiry_date_in_millis" : 1638987975422,
"max_nodes" : 1000,
"issued_to" : "elasticsearch",
"issuer" : "elasticsearch",
"start_date_in_millis" : -1
}
}
Thank you for your help

Symfony 4.2.8 : Can't connect to ES 7.10 through the fos:elastica:populate command

I encounter a problem in my production site in Symfony 4.2.8 (yes I know...).
When I try to launch a bin/console fos:elastica:populate or bin/console fos:elastica:reset the console crash (with -vvv option) :
Exception trace:
() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Transport/Http.php:190
Elastica\Transport\Http->exec() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Request.php:194
Elastica\Request->send() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Client.php:689
Elastica\Client->request() at /var/www/current/vendor/friendsofsymfony/elastica-bundle/src/Elastica/Client.php:58
FOS\ElasticaBundle\Elastica\Client->request() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Client.php:721
Elastica\Client->requestEndpoint() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Index.php:586
Elastica\Index->requestEndpoint() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Index.php:225
Elastica\Index->delete() at /var/www/current/vendor/ruflin/elastica/lib/Elastica/Index.php:296
Elastica\Index->create() at /var/www/current/vendor/friendsofsymfony/elastica-bundle/src/Index/Resetter.php:110
FOS\ElasticaBundle\Index\Resetter->resetIndex() at /var/www/current/vendor/friendsofsymfony/elastica-bundle/src/Command/PopulateCommand.php:184
FOS\ElasticaBundle\Command\PopulateCommand->populateIndex() at /var/www/current/vendor/friendsofsymfony/elastica-bundle/src/Command/PopulateCommand.php:164
FOS\ElasticaBundle\Command\PopulateCommand->execute() at /var/www/current/vendor/symfony/console/Command/Command.php:255
Symfony\Component\Console\Command\Command->run() at /var/www/current/vendor/symfony/console/Application.php:926
Symfony\Component\Console\Application->doRunCommand() at /var/www/current/vendor/symfony/framework-bundle/Console/Application.php:89
Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand() at /var/www/current/vendor/symfony/console/Application.php:269
Symfony\Component\Console\Application->doRun() at /var/www/current/vendor/symfony/framework-bundle/Console/Application.php:75
Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /var/www/current/vendor/symfony/console/Application.php:145
Symfony\Component\Console\Application->run() at /var/www/current/bin/console:39
In Http.php line 190:
[Elastica\Exception\Connection\HttpException]
Couldn't connect to host, Elasticsearch down?
A curl on ES server respond :
{
"name" : "prod",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "********",
"version" : {
"number" : "7.10.0",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "51e9d6f22758d0374a0f3f5c6e8f3a7997850f96",
"build_date" : "2020-11-09T21:30:33.964949Z",
"build_snapshot" : false,
"lucene_version" : "8.7.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
I create a new branch to upgrade all the component but I need a clue to operate a quick fix on production...
If somebody have an idea thank in advance.

Cloudera Parcel: Stoprunner not executing

I have build a parcel and a csd that go together and work fine when deploying to a cluster.
However, when I stop the service a child process started by a startup script keeps on running in the background.
I have tried many things but I went back to something more "brutal".
CSD extract
"startRunner" : {
"program" : "scripts/rexster.sh",
"args" : [ "start" ],
"environmentVariables" : {
"CONF_FILE" : "${conf_file}",
"REXSTER_PORT" : "${rexster_port}",
"HBASE_ZK" : "${hbase_zk_quorum}",
"REXSTER_SHUTDOWN_PORT" : "${rexster_shutdown_port}",
"HBASE_TABLE_NAME" : "${hbase_tablename}",
"MEM_XMX" : "${memory_xmx}",
"MEM_XMS" : "${memory_xms}"
}
},
"stopRunner" : {
"relevantRoleTypes" : ["TITAN_REXTER_SERVER"],
"runner" : {
"program" : "scripts/rexster.sh",
"args" : [ "stop" ]
}
}
}
scripts/rexster.sh extract
...
;;
(stop)
echo "Stopping rexster"
exec kill -9 `ps aux | grep java | grep titan | awk '{print $2}'`
# exec stopRexster.sh $TITAN_HOME
;;
(*)
echo "Don't understand [$CMD]"
;;
esac
But the process keeps on running in the background:
usr/java/jdk1.7.0_67-cloudera/bin/java -server -Xms128m -Xmx512m -Dtitan.logdir=../log com.tinkerpop.rexster.Application -s -c /opt/cloudera/parcels/TITAN-1.0/lib/titan-0.5.2-hadoop2-CDH5.3/conf/rexster-hbase.xml
The stoprunner wasn't placed correctly.
It is defined per role, after the role definition.
<...ROLE...>}]
,
"stopRunner" : {
"masterRole":"TITAN_REXTER_SERVER",
"relevantRoleTypes" : ["TITAN_REXTER_SERVER"],
"runner" : {
"program" : "scripts/rexster.sh",
"args" : [ "stop" ],
"environmentVariables" : {
"CONF_FILE" : "${conf_file}",
"REXSTER_PORT" : "${rexster_port}",
"HBASE_ZK" : "${hbase_zk_quorum}",
"REXSTER_SHUTDOWN_PORT" : "${rexster_shutdown_port}",
"HBASE_TABLE_NAME" : "${hbase_tablename}"
}
}
}

Resources