I want to connect RPC connection using java instead of kotlin. In corda 4.0 they have given using kotlin but I need it in java.
I have tried using java but its not working...
#RestController
#RequestMapping("/api/example/")
below code snippet from WebFlux module which connects to corda node using RPC:
#Component
public class NodeRPCConnection implements AutoCloseable {
// The host of the node we are connecting to.
#Value("${config.rpc.host}")
private String host;
// The RPC port of the node we are connecting to.
#Value("${config.rpc.username}")
private String username;
// The username for logging into the RPC client.
#Value("${config.rpc.password}")
private String password;
// The password for logging into the RPC client.
#Value("${config.rpc.port}")
private int rpcPort;
private CordaRPCConnection rpcConnection;
private CordaRPCOps proxy;
#PostConstruct
public void initialiseNodeRPCConnection() {
NetworkHostAndPort rpcAddress = new NetworkHostAndPort(host, rpcPort);
CordaRPCClient rpcClient = new CordaRPCClient(rpcAddress);
rpcConnection = rpcClient.start(username, password);
proxy = rpcConnection.getProxy();
}
#PreDestroy
public void close() {
rpcConnection.notifyServerAndClose();
}
public CordaRPCOps getProxy() {
return proxy;
}
}
Related
I have difficulty understanding some Kafka concepts in Java Spring Boot. I’d like to test a consumer against a real Kafka broker running on a server, which has some producers that write / have already written data to various topics. I would like to establish a connection with the server, consume the data, and verify or process its content in a test.
An enormous majority of examples (actually all I have seen so far) in the internet refer to embedded kafka, EmbeddedKafkaBroker, and show both a producer and a consumer implemented on one machine, locally. I haven’t found any example that would explain how to make a connection with a remote kafka server and read data from a particular topic.
I've written some code and I've printed the broker address with:
System.out.println(embeddedKafkaBroker.getBrokerAddress(0));
What I got is 127.0.0.1:9092, which means that it is local, so the connection with the remote server has not been established.
On the other hand, when I run the SpringBootApplication I get the payload from the remote broker.
Receiver:
#Component
public class Receiver {
private static final String TOPIC_NAME = "X";
private static final Logger LOGGER = LoggerFactory.getLogger(Receiver.class);
private CountDownLatch latch = new CountDownLatch(1);
public CountDownLatch getLatch() {
return latch;
}
#KafkaListener(topics = TOPIC_NAME)
public void receive(final byte[] payload) {
LOGGER.info("received the following payload: '{}'", payload);
latch.countDown();
}
}
Config:
#EnableKafka
#Configuration
public class ByteReceiverConfig {
#Autowired
EmbeddedKafkaBroker kafkaEmbeded;
#Value("${spring.kafka.bootstrap-servers}")
private String bootstrapServers;
#Value("${spring.kafka.consumer.group-id}")
private String groupIdConfig;
#Bean
public KafkaListenerContainerFactory<?> kafkaListenerContainerFactory() {
final ConcurrentKafkaListenerContainerFactory<Object, Object> factory =
new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory());
return factory;
}
#Bean
ConsumerFactory<Object, Object> consumerFactory() {
return new DefaultKafkaConsumerFactory<>(consumerProperties());
}
#Bean
Map<String, Object> consumerProperties() {
final Map<String, Object> properties =
KafkaTestUtils.consumerProps("junit-test", "true", this.kafkaEmbeded);
properties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
properties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class);
properties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class);
properties.put(ConsumerConfig.GROUP_ID_CONFIG, groupIdConfig);
return properties;
}
Test:
#EnableAutoConfiguration
#EnableKafka
#SpringBootTest(classes = {ByteReceiverConfig.class, Receiver.class})
#EmbeddedKafka
#ContextConfiguration(classes = ByteReceiverConfig.class)
#TestPropertySource(properties = { "spring.kafka.bootstrap-servers=${spring.embedded.kafka.brokers}",
"spring.kafka.consumer.group-id=EmbeddedKafkaTest"})
public class KafkaTest {
#Autowired
private KafkaListenerEndpointRegistry kafkaListenerEndpointRegistry;
#Autowired
EmbeddedKafkaBroker embeddedKafkaBroker;
#Autowired
Receiver receiver;
#BeforeEach
void waitForAssignment() {
for (MessageListenerContainer messageListenerContainer : kafkaListenerEndpointRegistry.getListenerContainers()) {
System.out.println(messageListenerContainer.getAssignedPartitions().isEmpty());
System.out.println(messageListenerContainer.toString());
System.out.println(embeddedKafkaBroker.getTopics().size());
System.out.println(embeddedKafkaBroker.getPartitionsPerTopic());
System.out.println(embeddedKafkaBroker.getBrokerAddress(0));
System.out.println(embeddedKafkaBroker.getBrokersAsString());
ContainerTestUtils.waitForAssignment(messageListenerContainer,
embeddedKafkaBroker.getPartitionsPerTopic());
}
#Test
public void testReceive() {
}
}
I would like somebody to shed some light on the following issues:
1.Can an instance of the class EmbeddedKafkaBroker be used to test data that comes from a remote broker, or is it only used for local tests, in which I would procude i.e send data to a topic that I created and consume data myself?
2.Is it possible to write a test class for a real kafka server? For instance to verify if a connection has been establish, or if a data has been read from a specific topic. What annotations, configurations, and classes would be needed in such case?
3.If I only want to consume data, do I have to provide the producer configuration in a config file (it would be strange, but all examples I have encountered so far did it)?
4.Do you know any resources (books, websites etc.) that show real examples of using kafka i.e. with a remote kafka server, with a procuder or a consumer only?
You don't need an embedded broker at all if you want to talk to an external broker only.
Yes, just set the bootstrap servers property appropriately.
No, you don't need producer configuration.
EDIT
#SpringBootApplication
public class So56044105Application {
public static void main(String[] args) {
SpringApplication.run(So56044105Application.class, args);
}
#Bean
public NewTopic topic() {
return new NewTopic("so56044105", 1, (short) 1);
}
}
spring.kafka.bootstrap-servers=10.0.0.8:9092
spring.kafka.consumer.enable-auto-commit=false
#RunWith(SpringRunner.class)
#SpringBootTest(classes = { So56044105Application.class, So56044105ApplicationTests.Config.class })
public class So56044105ApplicationTests {
#Autowired
public Config config;
#Test
public void test() throws InterruptedException {
assertThat(config.latch.await(10, TimeUnit.SECONDS)).isTrue();
assertThat(config.received.get(0)).isEqualTo("foo");
}
#Configuration
public static class Config implements ConsumerSeekAware {
List<String> received = new ArrayList<>();
CountDownLatch latch = new CountDownLatch(3);
#KafkaListener(id = "so56044105", topics = "so56044105")
public void listen(String in) {
System.out.println(in);
this.received.add(in);
this.latch.countDown();
}
#Override
public void registerSeekCallback(ConsumerSeekCallback callback) {
}
#Override
public void onPartitionsAssigned(Map<TopicPartition, Long> assignments, ConsumerSeekCallback callback) {
System.out.println("Seeking to beginning");
assignments.keySet().forEach(tp -> callback.seekToBeginning(tp.topic(), tp.partition()));
}
#Override
public void onIdleContainer(Map<TopicPartition, Long> assignments, ConsumerSeekCallback callback) {
}
}
}
There are some examples in this repository for bootstrapping real Kafka producers and consumers across a variety of configurations — plaintext, SSL, with and without authentication, etc.
Note: the repo above contains examples for the Effective Kafka book, which I am the author of. However, they can be used freely without the book and hopefully they make just as much sense on their own.
More to the point, here are a pair of examples for a basic producer and a consumer.
/** A sample Kafka producer. */
import static java.lang.System.*;
import java.util.*;
import org.apache.kafka.clients.producer.*;
import org.apache.kafka.common.serialization.*;
public final class BasicProducerSample {
public static void main(String[] args) throws InterruptedException {
final var topic = "getting-started";
final Map<String, Object> config =
Map.of(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092",
ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName(),
ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName(),
ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, true);
try (var producer = new KafkaProducer<String, String>(config)) {
while (true) {
final var key = "myKey";
final var value = new Date().toString();
out.format("Publishing record with value %s%n",
value);
final Callback callback = (metadata, exception) -> {
out.format("Published with metadata: %s, error: %s%n",
metadata, exception);
};
// publish the record, handling the metadata in the callback
producer.send(new ProducerRecord<>(topic, key, value), callback);
// wait a second before publishing another
Thread.sleep(1000);
}
}
}
}
/** A sample Kafka consumer. */
import static java.lang.System.*;
import java.time.*;
import java.util.*;
import org.apache.kafka.clients.consumer.*;
import org.apache.kafka.common.serialization.*;
public final class BasicConsumerSample {
public static void main(String[] args) {
final var topic = "getting-started";
final Map<String, Object> config =
Map.of(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092",
ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName(),
ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName(),
ConsumerConfig.GROUP_ID_CONFIG, "basic-consumer-sample",
ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest",
ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false);
try (var consumer = new KafkaConsumer<String, String>(config)) {
consumer.subscribe(Set.of(topic));
while (true) {
final var records = consumer.poll(Duration.ofMillis(100));
for (var record : records) {
out.format("Got record with value %s%n", record.value());
}
consumer.commitAsync();
}
}
}
}
Now, these are obviously not unit tests. But with very little rework they could be turned into one. The next step would be to remove Thread.sleep() and add assertions. Note, since Kafka is inherently asynchronous, naively asserting a published message in a consumer immediately after publishing will fail. For a robust, repeatable test, you may want to use something like Timesert.
I have created a CordaService running on my node. I want this service to start flows based on various conditions. However, the ServiceHub provided to the service does not provide the ability to start flows.
Is there any flow for a service to start a flow? How would I do thi?
Yes. Simply pass your CordaService an AppServiceHub instead of a ServiceHub in its constructor.
The AppServiceHub interface extends the ServiceHub interface to give a node the ability to start flows:
interface AppServiceHub : ServiceHub {
fun <T> startFlow(flow: FlowLogic<T>): FlowHandle<T>
fun <T> startTrackedFlow(flow: FlowLogic<T>): FlowProgressHandle<T>
}
Yes, pass AppServiceHub to the constructor
In Kotlin:
class MyCordaService(private val serviceHub: AppServiceHub) : SingletonSerializeAsToken() {
init {
// code ran at service creation / node startup
}
// public api of service
}
or Java:
public class MyCordaService extends SingletonSerializeAsToken {
private AppServiceHub serviceHub;
public MyCordaService(AppServiceHub serviceHub) {
this.serviceHub = serviceHub;
// code ran at service creation / node startup
}
// public api of service
}
Important: to avoid possible any potential deadlocks between running nodes, start inner flows from their own Thread
For example:
public class MyCordaService extends SingletonSerializeAsToken {
private AppServiceHub serviceHub;
public MyCordaService(AppServiceHub serviceHub) {
this.serviceHub = serviceHub;
// code ran at service creation / node startup
}
// public api of service
public void doSomething(){
// do something and start a new flow
Thread flowThread = new Thread(new StartFlow());
flowThread.start();
}
private class StartFlow implements Runnable {
#Override
public void run() {
// start new flow
CordaFuture<SignedTransaction> cordaFuture= appServiceHub.startFlow(new
Flow(params).getReturnValue();
SignedTransaction signedTransaction = cordaFuture.get();
}
}
}
I am trying to create a spring resource server secured with oauth2.
I am using auth0 for my auth2 service, and I have an api and client configured with scopes.
I have a resource server that mostly works. It is secured, and I can use #EnableGlobalMethodSecurity and #PreAuthorize("#oauth2.hasScope('profile:read')") to limit access to tokens with that scope.
However, when I try to get the Principal or the OAuth2Authentication they are both null. I've configured the resource server to use the JWK key-set-uri.
I suspect that this has to do with the DefaultUserAuthenticationConverter trying to read the the 'user_name' claim form the JWT, but it needs to be reading it from the 'sub' claim, and I don't know how to change this behaviour.
First create a UserAuthenticationConverter:
public class OidcUserAuthenticationConverter implements UserAuthenticationConverter {
final String SUB = "sub";
#Override
public Map<String, ?> convertUserAuthentication(Authentication userAuthentication) {
throw new UnsupportedOperationException();
}
#Override
public Authentication extractAuthentication(Map<String, ?> map) {
if (map.containsKey(SUB)) {
Object principal = map.get(SUB);
Collection<? extends GrantedAuthority> authorities = null;
return new UsernamePasswordAuthenticationToken(principal, "N/A", authorities);
}
return null;
}
}
Then configure spring to use it like so:
#Configuration
public class OidcJwkTokenStoreConfiguration {
private final ResourceServerProperties resource;
public OidcJwkTokenStoreConfiguration(ResourceServerProperties resource) {
this.resource = resource;
}
#Bean
public TokenStore jwkTokenStore() {
DefaultAccessTokenConverter tokenConverter = new DefaultAccessTokenConverter();
tokenConverter.setUserTokenConverter(new OidcUserAuthenticationConverter());
return new JwkTokenStore(this.resource.getJwk().getKeySetUri(), tokenConverter);
}
}
I am using spring-amqp to consume messages from RabbitMQ in my web application.
Web application consists of multiple components in it such as (Redis, OracleDB)
Now i have a scenario, if any exception occurs due to infrastructure like Oracle server is down, Redis connection issue, i want to push message back to the same queue and after certain specified delay i want to consume the message back.
After certain delay then also the message is leading to same exception, probably i want to use maximum attempts option or do the same as above push the message back to queue and send a mail to administrator stating "Infrastructure Issue".
Does Spring AMQP supports above scenario.? If yes please provide me how to come up with such or similar solutions.
I tried below piece of code. Message is not going for dead letter queue instead it is re-queuing to same queue causing infinite loop. Please correct me where am i going wrong
Configuration class
#Configuration
public class MQConfig {
public static final String OUTGOING_QUEUE = "my.outgoing.example";
public static final String INCOMING_QUEUE = "my.incoming.example";
public static final String DEAD_LETTER_QUEUE = "my.deadletter.queue.example";
#Autowired
private ConnectionFactory cachingConnectionFactory;
// Setting the annotation listeners to use the jackson2JsonMessageConverter
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(cachingConnectionFactory);
factory.setMessageConverter(jackson2JsonMessageConverter());
factory.setDefaultRequeueRejected(false);
return factory;
}
// Standardize on a single objectMapper for all message queue items
#Bean
public Jackson2JsonMessageConverter jackson2JsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public Queue outgoingQueue() {
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-dead-letter-exchange", "dlx");
args.put("x-dead-letter-routing-key", DEAD_LETTER_QUEUE);
args.put("x-message-ttl", 50000);
return new Queue(OUTGOING_QUEUE, false, false, false, args);
}
#Bean
public RabbitTemplate outgoingSender() {
RabbitTemplate rabbitTemplate = new RabbitTemplate(cachingConnectionFactory);
rabbitTemplate.setQueue(outgoingQueue().getName());
// rabbitTemplate.setRoutingKey(outgoingQueue().getName());
rabbitTemplate.setMessageConverter(jackson2JsonMessageConverter());
return rabbitTemplate;
}
#Bean
public Queue incomingQueue() {
return new Queue(INCOMING_QUEUE);
}
#Bean
public Queue deadLetterQueue() {
return new Queue(DEAD_LETTER_QUEUE);
}
#Bean
public DirectExchange dlx() {
return new DirectExchange(DEAD_LETTER_QUEUE);
}
#Bean
public Binding dlqBinding() {
return BindingBuilder.bind(deadLetterQueue()).to(dlx()).with(DEAD_LETTER_QUEUE);
}
}
Core logic
#Component
public class DeadLetterSendReceive {
private static final Logger LOGGER = LoggerFactory.getLogger(DeadLetterSendReceive.class);
#Autowired
private RabbitTemplate outgoingSender;
// Scheduled task to send an object every 5 seconds
#Scheduled(fixedDelay = 5000)
public void sender() {
Integer int1[] = new Integer[]{10,20,30,40,50};
for (int i = 0; i <= int1.length; i++){
System.out.println(int1[i]);
if(int1[i]/10 == 1){
throw new AmqpRejectAndDontRequeueException("to deadletter queue");
}
else{
ExampleObject ex = new ExampleObject();
ex.setValue(int1[i]);
LOGGER.info("Sending example object at " + ex.getValue());
outgoingSender.convertAndSend(ex);
}
}
}
// Annotation to listen for an ExampleObject
#RabbitListener(queues = MQConfig.INCOMING_QUEUE)
public void handleMessage(ExampleObject exampleObject) {
LOGGER.info("Received incoming object at " + exampleObject.getValue());
}
}
Pojo Class
import java.util.Date;
public class ExampleObject {
private Date date = new Date();
private int value;
public int getValue() {
return value;
}
public void setValue(int value) {
this.value = value;
}
public ExampleObject() {
}
#Override
public String toString() {
return "ExampleObject{" +
"date= " + date +
'}';
}
public Date getDate() {
return date;
}
public void setDate(Date date) {
this.date = date;
}
}
There are a couple of ways to do it; use the delayed message exchange plugin and publish the failed message to it. You can set a header to track how many attempts have been made.
Or you can do it with a dead letter queue with a TTL where the dead-letter queue is configured with dead-lettering to send the expired message back to the original queue. See my answer to this question and its link to another answer.
You can use the x-death header to track retries; it has been changed in recent brokers to now keep a count instead of keep adding new entries to the header.
To force the message to go to the DLQ, set defaultRequeueRejected to false or throw an AmqpRejectAndDontRequeueException.
I am working in a Spring-MVC application and I would like to include personalized chat as a feature in it. After some research I found out Cometd to be a suitable option. After going through the documentation and forever repeating samples, I have a little bit of setup which I have done. I need some help to integrate a personalized chat service in the spring-mvc app, and enabling private chat when user pushes chat button.
So basically, I found out, "/service/chat" can be used for private chat, so I have a class for that, and to use private chat, I must have a mapping of userid<-->sessionId, but I cannot find examples anywhere how to do it. I am posting some of the code I have, kindly let me know what is remaining to do, and if possible, some resources, samples for that.
Controller code:
#Controller
#Singleton
public class MessageController {
private MessageService messageService;
#Autowired(required = true)
#Qualifier(value ="messageService")
public void setMessageService(MessageService messageService){this.messageService=messageService;}
#RequestMapping(value = "/startchatting", produces = "application/text")
#ResponseBody
public String startChattingService(){
return "OK";
}
#RequestMapping(value = "/stopchatting",produces = "application/text")
#ResponseBody
public String stopChatting(){
return "OK";
}
}
Private Message Service :
#Service
public class PrivateMessageService {
#Session
private ServerSession session;
#Listener("/service/private")
public void handlePrivateMessage(ServerSession sender, ServerMessage message){
String userId = (String) message.get("targetUserId");
//Mapping code necessary to map userids to session-id's.
//ServerSession recipient = findServerSessionFromUserId(userId);
//recipient.deliver(session,message.getChannel(),message.getData(),null);
}
}
CometConfigurer :
#Component
#Singleton
public class CometConfigurer {
private BayeuxServer bayeuxServer;
private ServerAnnotationProcessor processor;
#Inject
public void setBayeuxServer(BayeuxServer bayeuxServer){this.bayeuxServer = bayeuxServer;}
#PostConstruct
public void init() {this.processor= new ServerAnnotationProcessor(bayeuxServer);}
public Object postProcessBeforeInitialization(Object bean, String name) throws BeansException {
System.out.println("Configuring service " + name);
processor.processDependencies(bean);
processor.processConfigurations(bean);
processor.processCallbacks(bean);
return bean;
}
public Object postProcessAfterInitialization(Object bean, String name) throws BeansException {
return bean;
}
public void postProcessBeforeDestruction(Object bean, String name) throws BeansException {
processor.deprocessCallbacks(bean);
}
#Bean(initMethod = "start", destroyMethod = "stop")
public BayeuxServer bayeuxServer() {
BayeuxServerImpl bean = new BayeuxServerImpl();
// bean.setOption(BayeuxServerImpl.LOG_LEVEL, "3");
return bean;
}
public void setServletContext(ServletContext servletContext) {
servletContext.setAttribute(BayeuxServer.ATTRIBUTE, bayeuxServer);
}
}
Cometd beans :
<beans:bean id="bayeuxServer" class="org.cometd.server.BayeuxServerImpl" init-method="start" destroy-method="stop"/>
I have directly included the JSP files which have cometd configuration and setup from https://github.com/fredang/cometd-spring-example, and modified them to serve my needs. Kindly let me know what else is remaining, all suggestions are welcome, I am unable to find any examples for same task on net, which are detailed, and have more code then explanation. Thank you.
Using Spring 4.x's new WebSocket feature would definitely work; moreover, this new module ships with lots of very interesting features for your use case:
STOMP protocol support
messaging abstractions
session management
pub/sub mechanisms
etc
You can check this nice chat application that demonstrates all those features.