Ethereum Chainlink HTTP Get not pinging my HTTP endpoint - http

I am attempting to have my Ethereum smart contract connect to an external HTTP endpoint using Chainlink. Following along with Chainlink's documentation (https://docs.chain.link/docs/advanced-tutorial/) I deployed this contract onto the Rinkeby testnet.
pragma solidity ^0.8.7;
import "github.com/smartcontractkit/chainlink/blob/develop/contracts/src/v0.8/ChainlinkClient.sol";
// MyContract inherits the ChainlinkClient contract to gain the
// functionality of creating Chainlink requests
contract getHTTP is ChainlinkClient {
using Chainlink for Chainlink.Request;
bytes32 private thisDoesNotWork;
address private owner;
address private ORACLE_ADDRESS = 0x718Cc73722a2621De5F2f0Cb47A5180875f62D60;
bytes32 private JOBID = stringToBytes32("86b489ec4d84439c96181a8df7b22223");
string private url = "<myHTTPAddressAsString>";
// This endpoint URL is hard coded in my contract, and stored as a string (as in the example code).
// I control it and can have it reply with whatever I want, which might be an issue, returning data in a format that the oracle rejects
uint256 constant private ORACLE_PAYMENT = 100000000000000000;
constructor() public {
// Set the address for the LINK token for the network
setPublicChainlinkToken();
owner = msg.sender;
}
function requestBytes()
public
onlyOwner
{
Chainlink.Request memory req = buildChainlinkRequest(JOBID, address(this), this.fulfill.selector);
req.add("get", url);
sendChainlinkRequestTo(ORACLE_ADDRESS, req, ORACLE_PAYMENT);
}
function fulfill(bytes32 _requestId, bytes32 recVal)
public
recordChainlinkFulfillment(_requestId)
{
thisDoesNotWork = recVal;
}
function cancelRequest(
bytes32 _requestId,
uint256 _payment,
bytes4 _callbackFunctionId,
uint256 _expiration
)
public
onlyOwner
{
cancelChainlinkRequest(_requestId, _payment, _callbackFunctionId, _expiration);
}
// withdrawLink allows the owner to withdraw any extra LINK on the contract
function withdrawLink()
public
onlyOwner
{
LinkTokenInterface link = LinkTokenInterface(chainlinkTokenAddress());
require(link.transfer(msg.sender, link.balanceOf(address(this))), "Unable to transfer");
}
modifier onlyOwner() {
require(msg.sender == owner);
_;
}
// A helper funciton to make the string a bytes32
function stringToBytes32(string memory source) private pure returns (bytes32 result) {
bytes memory tempEmptyStringTest = bytes(source);
if (tempEmptyStringTest.length == 0) {
return 0x0;
}
assembly { // solhint-disable-line no-inline-assembly
result := mload(add(source, 32))
}
}
}
I found a node on the Chainlink market (https://market.link/jobs/529c7194-c665-4b30-8d25-5321ea49d9cc) that is currently active on rinkeby (according to Etherscan it has been active within the past 3 days and presumably still working).
I deploy the contract and fund the contract with LINK. I call the requestBytes() function through remix and everything works as expected. Metamask pays the gas, the LINK is removed from my contract, I get a transaction hash, and no errors.
However, my endpoint never logs a request attempt, the oracle never lists a transaction on its Etherscan page, and my data is not present.
I have attempted to use other jobs from the Chainlink market with similar outcomes.
I have also attempted to use other HTTP endpoints, like the ones from the Chainlink examples, with similar outcomes, however I doubt this is the issue, since it appears the HTTP request is never even getting called (as referenced by the fact that my HTTP endpoint does not log the request)
Without an error message, and being new to Web3 dev, I am not sure where to start debugging. I found this comment on Github: https://github.com/smartcontractkit/documentation/issues/513 and implemented the suggestion here without luck.
I also found this: Chainlink - Job not being fulfilled but this was not helpful either.
My current considerations for where the error might be:
The oracles are whitelisted and reject my request outright. Have considered creating my own node but want to avoid if possible at this stage.
I have an type error in how I am formatting the request in my contract, like the example in the GitHub exchange I found and referenced above.
EDIT: I am also open to other options beyond Chainlink to connect my contract to an HTTP GET endpoint, if anyone has any suggestions. Thanks!

I've been working on something similar recently and would suggest you try using the kovan network and the oracle that chainlink has there. Even more specifically, I think it would be a good idea to confirm you can get it working using the api, oracle, and jobid listed in the example on that page you are following... here:
https://docs.chain.link/docs/advanced-tutorial/#contract-example
Once you get that example working, then you can modify it for your usage. The jobid in that tutorial is for returning a (multiplied) uint256... which, for your API, I think is not what you want as you are wanting bytes32 it sounds like... so when you try to use it with your API that returns bytes32 the jobid would be: 7401f318127148a894c00c292e486ffd as seen here:
https://docs.chain.link/docs/decentralized-oracles-ethereum-mainnet/
Another thing that might be your issue, is your api. You say you control what it returns... I think it might have to return a response in bytes format, like Patrick says in his response (and his comments on his response) here:
Get a string from any API using Chainlink Large Response Example
Hope this is helpful. If you cannot get the example in the chainlink docs to work, let me know.

Related

How to forward incoming data via REST to an SSE stream in Quarkus

In my setting I want to forward certain status changes via an SSE channel (Server sent events). The status changes are initiated by calling a REST endpoint. So, I need to forward the incoming status change to the SSE stream.
What is the best/simplest way to accomplish this in Quarkus.
One solution I can think of is to use an EventBus (https://quarkus.io/guides/reactive-messaging). The SSE endpoint would subscribe to the status changes and push it through the SSE channel. The status change endpoint publishes appropriate events.
Is this a viable solution? Are there other (simpler) solutions? Do I need to use the reactive stuff in any case to accomplish this?
Any help is very appreciated!
Easiest way would be to use rxjava as a stream provider. Firstly you need to add rxjava dependency. It can go either from reactive dependencies in quarkus such as kafka, or by using it directly(if you don't need any streaming libraries):
<dependency>
<groupId>io.reactivex.rxjava2</groupId>
<artifactId>rxjava</artifactId>
<version>2.2.19</version>
</dependency>
Here's example on how to send random double value each second:
#GET
#Path("/stream")
#Produces(MediaType.SERVER_SENT_EVENTS)
#SseElementType("text/plain")
public Publisher<Double> stream() {
return Flowable.interval(1, TimeUnit.SECONDS).map(tick -> new Random().nextDouble());
}
We create new Flowable which will fire every second and on each tick we generate next random double. Investigate any other options on how you can create Flowable such as Flowable.fromFuture() to adapt it for your specific code logic.
P.S code above will generate new Flowable each time you query this endpoint, I made it to save up space, in your case I assume you'll have a single source of events that you can build once and use the same instance every time endpoint queried
Dmytro, thanks for pointing me in the right direction.
I have opted for Mutiny in connection with Kotlin. My code now looks like this:
data class DeviceStatus(var status: Status = Status.OFFLINE) {
enum class Status {OFFLINE, CONNECTED, ANALYZING, MAINTENANCE}
}
#ApplicationScoped
class DeviceStatusService {
var deviceStatusProcessor: PublishProcessor<DeviceStatus> = PublishProcessor.create()
var deviceStatusQueue: Flowable<DeviceStatus> = Flowable.fromPublisher(deviceStatusProcessor)
fun pushDeviceStatus(deviceStatus: DeviceStatus) {
deviceStatusProcessor.onNext(deviceStatus)
}
fun getStream(): Multi<DeviceStatus> {
return Multi.createFrom().publisher(deviceStatusQueue)
}
}
#Path("/deviceStatus")
class DeviceStatusResource {
private val LOGGER: Logger = Logger.getLogger("DeviceStatusResource")
#Inject
#field: Default
lateinit var deviceStatusService: DeviceStatusService
#POST
#Consumes(MediaType.APPLICATION_JSON)
fun status(status: DeviceStatus): Response {
LOGGER.info("POST /deviceStatus " + status.status);
deviceStatusService.pushDeviceStatus(status)
return Response.ok().build();
}
#GET
#Path("/eventStream")
#Produces(MediaType.SERVER_SENT_EVENTS)
#SseElementType(MediaType.APPLICATION_JSON)
fun stream(): Multi<DeviceStatus>? {
return deviceStatusService.getStream()
}
}
As minimal setup the service could directly use the deviceStatusProcessor as publisher. However, the Flowable adds buffering.
Comments on the implementation are welcome.

gRPC: How can I distinguish bi-streaming clients at server side?

In this tutorial and example code, a server can call onNext() method on every stream observer, which will broadcast messages to all clients bi-streaming with the server. But there is no method to identify which observer corresponds to which client. How can a server push a message to specific client instead of broadcasting?
According to this answer it is possible to map each observer if client id is provided by metadata. It seems const auto clientMetadata = context->client_metadata(); part does the trick, but I'm working with Java, not C++. Are there any Java equivalent for getting the metadata at server side?
The answer depends a bit on how the clients will be identified. If the initial request provided a handle (like a username, but not registered ahead-of-time), then you could just wait for the first onNext():
public StreamObserver<Chat.ChatMessage> chat(StreamObserver<Chat.ChatMessageFromServer> responseObserver) {
return new StreamObserver<Chat.ChatMessage>() {
#Override
public void onNext(Chat.ChatMessage value) {
String userHandle = value.getHandle();
// observers would now be a map, not a set
observers.put(userHandle, responseObserver);
...
Let's say instead that all users are logged in, and provide a token in the headers, like OAuth. Then you would use an interceptor to authenticate the user and Context to propagate it to the application, as in https://stackoverflow.com/a/40113309/4690866 .
public StreamObserver<Chat.ChatMessage> chat(StreamObserver<Chat.ChatMessageFromServer> responseObserver) {
// USER_IDENTITY is a Context.Key, also used by the interceptor
User user = USER_IDENTITY.get();
observers.put(user.getName(), responseObserver);
return new StreamObserver<Chat.ChatMessage>() {
...
The first one is easier/nicer when the identification only applies to this one RPC. The second one is easier/nicer when the identification applies to many RPCs.

Android <-> iOS Bluetooth LE application, can't write to characteristic

I'm developing a ble-based native local multiplayer plugin for Unity (for both Android and iOS). I use a single service, with a single characteristic with rw permissions. I've managed to make Android<->Android and iOS<->iOS work all right, but I'm having a rough time trying to make Android<->iOS work. Specifically, it's the 'iOS as Peripheral, Android as Central' combination the one that keeps me up at night. After many hours of fiddling, testing, googling and trying, I have very much pinned down the problem to this:
From the Android side, if I don't subscribe to the characteristic, a call to BluetoothGatt#writeCharacteristic(characteristic), like this:
String str = "the data";
xferCharacteristic.setValue(str.getBytes("UTF-8"));
mGatt.writeCharacteristic(xferCharacteristic);
will return 'true' and succeed, and the peripheralManager:didReceiveWriteRequests: callback will be called on the iOS side where I can manipulate the precious received data as I see fit. So far so good. But, if I try to update a characteristic from the iOS end, the Android central won't get notified (the callback BluetoothGattCallback#onCharacteristicChanged should be called, but it isn't), since it did not subscribe to the characteristic.
If I make the Android central subscribe to the characteristic offered by the iOS peripheral, by means of this section of code:
First, connect to the iOS peripheral with
public void onScanResult(int callbackType, ScanResult result) {
BluetoothDevice btDevice = result.getDevice();
mGatt = device.connectGatt(appContext, false, mGattCallback);
...
with mGattCallback an instance of BLEGattCallback which will handle the onServicesDiscovered callback:
public class BLEGattCallback extends BluetoothGattCallback {
private static final UUID CLIENT_CHARACTERISTIC_CONFIG_DESCRIPTOR_UUID = UUID.fromString("00002902-0000-1000-8000-00805f9b34fb");
public void onServicesDiscovered(BluetoothGatt gatt, int status) {
List<BluetoothGattService> services = gatt.getServices();
for(BluetoothGattService s : services) { // foreach service...
if(UUID.fromString(MyServiceUUID).equals(s.getUuid())) { // just the one I want...
List<BluetoothGattCharacteristic> characteristics = s.getCharacteristics();
for(BluetoothGattCharacteristic c : characteristics) { // foreach characteristic...
if(c.getUuid().toString().equals(BLEManager.FGUUIDXferQueueString)) { // just the char. I want...
c.setWriteType(BluetoothGattCharacteristic.WRITE_TYPE_NO_RESPONSE);
for (BluetoothGattDescriptor descriptor : c.getDescriptors()) {
if(descriptor.getUuid().equals(CLIENT_CHARACTERISTIC_CONFIG_DESCRIPTOR_UUID)) {
descriptor.setValue(BluetoothGattDescriptor.ENABLE_NOTIFICATION_VALUE);
gatt.writeDescriptor(descriptor);
}
}
gatt.setCharacteristicNotification(c, true);
}
}
}
}
}
This makes the Android central correctly subscribe for the characteristic (the callback method peripheralManager:central:didSubscribeToCharacteristic: is called on the iOS peripheral), BUT, if i do this, the call to mGatt.writeCharacteristic(xferCharacteristic) will return 'false' and no data will be written to the peripheral, so it's a kind of can-only-write or can-only-notify-update situation.
I have unsuccessfully tried to find out the meaning of writeCharacteristic returning 'false', to no avail (seriously, an error code would help a lot).
I've tried a lot of different combinations, values, etc... but, bottom line: as soon as I call gatt.writeDescriptor subsequent calls to writeCharacteristic will fail, and if I don't call gatt.writeDescriptor the android central won't subscribe.
I'm pretty much stuck here. Any help appreciated. Thanks a lot.
Classic issue. You must wait for the operation to complete before you can issue another one. See Android BLE BluetoothGatt.writeDescriptor() return sometimes false.
Thanks to the received hint, this issue has been solved. These are the changes I made to the code:
The Android client must wait for the writeDescriptor(...) request to finish before issuing a writeCharacteristic(...) command. For that, I had to #Override the method onDescriptorWrite on my BLEGattCallback class, which will be called when the writeDescriptor operation completes. I moved my first writeCharacteristic(...) call here, and now the information is sent to the iOS endpoint (the rest must be flow-controlled). So I'm very happy.

Implementing asynchronous publish subscribe topic in tibco ems

my scenario is that me as a movie distributor, need to update my clients on new movies, I publish this information on a topic with durable subscribers and clients who want to buy the movie will express their interest.
However, this is where things go south, my implementation of the publisher stops listening as soon as it receives the first reply. Any help would be greatly appreciated. Thank you.
request(Message message)
Sends a request and waits for a reply.
The temporary topic is used for the JMSReplyTo destination; the first reply is returned, and any following replies are discarded.
https://docs.oracle.com/javaee/6/api/javax/jms/TopicRequestor.html
First thing first... I have questions regarding the scenario. Is this some kind of test/exercice, or are we talking about a real world scenario ?
Are all client interested in the movie SEPARATE topic subscribers ? How does that scale ? I the plan to have a topic for every movie, and possible interested parties declaring durable subscribers (one each, for every movie) ? This seems to be abuse of durable subcribers... I would suggest using ONLY one subscriber (in system B) to a "Movie Released" event/topic (from system A), and have some code (in system B) reading all the clients from a DB to send emails/messages/whatever. (If system A and B are the same, it may or not be a good idea to use EMS at all... depends.)
If it is not an exercise, I must comment : Don't use a MOM (EMS, ActiveMQ) to do a DBMS' (Oracle, PostGreSQL) work !
With the disclaimer section done, I suggest an asynchronous subscription approach (These two clips are taken for the EMS sample directory. File tibjmsAsyncMsgConsumer.java).
Extract from the constructor (The main class must implements ExceptionListener, MessageListener):
ConnectionFactory factory = new com.tibco.tibjms.TibjmsConnectionFactory(serverUrl);
/* create the connection */
connection = factory.createConnection(userName,password);
/* create the session */
session = connection.createSession();
/* set the exception listener */
connection.setExceptionListener(this);
/* create the destination */
if (useTopic)
destination = session.createTopic(name);
else
destination = session.createQueue(name);
System.err.println("Subscribing to destination: "+name);
/* create the consumer */
msgConsumer = session.createConsumer(destination);
/* set the message listener */
msgConsumer.setMessageListener(this);
/* start the connection */
connection.start();
The method is then called every time a message arrives.
public void onMessage(Message msg)
{
try
{
System.err.println("Received message: " + msg);
}
catch (Exception e)
{
System.err.println("Unexpected exception in the message callback!");
e.printStackTrace();
System.exit(-1);
}
}
You want to continue reading messages in a loop. Here is an example:
/* read messages */
while (true)
{
/* receive the message */
msg = msgConsumer.receive();
if (msg == null)
break;
if (ackMode == Session.CLIENT_ACKNOWLEDGE ||
ackMode == Tibjms.EXPLICIT_CLIENT_ACKNOWLEDGE ||
ackMode == Tibjms.EXPLICIT_CLIENT_DUPS_OK_ACKNOWLEDGE)
{
msg.acknowledge();
}
System.err.println("Received message: "+ msg);
}
You may want to also consider a possible issue with durable consumers. If your consumers never pick up their messages, storage will continue to grow at the server side. For this reason you may want to send your messages with an a expiration time, and/or limit maximum number of messages (or size in KB/MB/GB) of the JMS topics you are using.

OpenRasta unit testing

I’m about to start work on an OpenRasta project (an xml over http web service). OpenRasta looks great but unfortunately worked examples seem few and far between on the internet. Looking at the test side of the project, if my handlers are returning strongly typed objects (not OperationResult), i.e.:
public class PersonHandler
...
public Person Get(int id)
{
...
How can I test for http status codes? (For example if the handler throws an uncaught exception). I’m not sure what level the tests pitch in at, and what needs mocking (using moq btw)
Any help appreciated, particularly coded examples!
I faced the same problem, and ended up writing my tests as integration tests at a much higher level, actually making real REST/HTTP calls through a simple HttpWebRequest client. This allowed me to check the HTTP response headers / status codes and double-check the JSON/XML serialization from the client's perspective, which was just as important as whether or not the operations succeeded.
I started by returning OperationResult from all my handlers, and used these to wrap the strongly-typed objects. My handlers all inherit from a base class with a few helper methods that make it easier to return a custom error with a user-friendly error message. The more I coded this up, the more my handlers resembled a ASP.NET MVC controller. e.g.:
public OperationResult GetById(int id)
{
try
{
// do stuff here
return OKResult( // some strongly-typed resource );
}
catch(SomeException ex)
{
return BadRequestResult(SomeErrorCode, ex.Message);
}
}
Then in the test client, it's pretty easy to just check the HTTP status code. Obviously this doesn't help much with mocking. I'm not sure what the best solution is, in fact I've favorited this question in the hope that someone answers it better than I can - but this has worked pretty well for me so far.
The handler is just a class--ideally with minimal dependencies--so your unit tests can just test the isolated logic in the class.
If you want to test for status codes, I recommend (based on very little experience!) using OpenRasta self-hosting.
Here's a test (somewhat changed) that I wrote recently:
[TestMethod]
public void POST_with_inaccurate_contentLength_returns_405()
{
var resource = GetResource();
IRequest request = new InMemoryRequest
{
HttpMethod = "POST",
Uri = new Uri("http://localhost/Resource"),
};
request.Headers.ContentLength = 16; //wrong!
request.Entity.Stream.Write(resource.Content, 0, resource.Content.Length);
var response = _host.ProcessRequest(request);
Assert.AreEqual(405, response.StatusCode);
}
I should add that the host is set up in the TestInitialize method as such:
_host = new InMemoryHost(new Configuration());
_host.Resolver.AddDependencyInstance(typeof(IFileResourceRepository), _repository, DependencyLifetime.Singleton);
...and is cleaned up in the TestCleanup method.

Resources