I am using kcl api version 2 ,and dont want to use Dynamodb Client for storing the records .
private static final Logger LOG = LoggerFactory.getLogger(DisplayConsumerApplication.class);
public static void main(String... args) {
KinesisAsyncClient kinesisClient = KinesisAsyncClient.builder().credentialsProvider(ProfileCredentialsProvider.create())
.region(Region.of("US-EAST-1")).build();
//DynamoDbAsyncClient dynamoClient =
// DynamoDbAsyncClient.builder().credentialsProvider(ProfileCredentialsProvider.
// create()) .region(Region.of("US-EAST-1")).build();
CloudWatchAsyncClient cloudWatchClient = CloudWatchAsyncClient.builder().credentialsProvider(ProfileCredentialsProvider.create())
.region(Region.of("US-EAST-1")).build();
ConfigsBuilder configsBuilder = new ConfigsBuilder("Sample","Sample", kinesisClient,null,
cloudWatchClient, UUID.randomUUID().toString(), new DisplayConsumerFactory());
Scheduler scheduler = new Scheduler(configsBuilder.checkpointConfig(), configsBuilder.coordinatorConfig(),
configsBuilder.leaseManagementConfig(), configsBuilder.lifecycleConfig(), configsBuilder.metricsConfig(),
configsBuilder.processorConfig(), configsBuilder.retrievalConfig());
Thread schedulerThread = new Thread(scheduler);
schedulerThread.setDaemon(true);
schedulerThread.start();
System.out.println("Press enter to shutdown");
BufferedReader reader = new BufferedReader(new InputStreamReader(System.in));
try {
reader.readLine();
} catch (IOException ioex) {
LOG.error("Caught exception while waiting for confirm. Shutting down", ioex);
}
Future<Boolean> gracefulShutdownFuture = scheduler.startGracefulShutdown();
LOG.info("Waiting up to 20 seconds for shutdown to complete.");
try {
gracefulShutdownFuture.get(20, TimeUnit.SECONDS);
} catch (InterruptedException e) {
LOG.info("Interrupted while waiting for graceful shutdown. Continuing.");
} catch (ExecutionException e) {
LOG.error("Exception while executing graceful shutdown.", e);
} catch (TimeoutException e) {
LOG.error("Timeout while waiting for shutdown. Scheduler may not have exited.");
}
LOG.info("Completed, shutting down now.");
}
}
As you can see I commented initialize of DynamodbClient ,but in that method it is manadatory to pass the object of Dynamoclient .So I passed as null ,but getting null pointer exception ,Could you please share your idea how I can use Scheduler without dynamodb client ?
Related
On starting, the application with SignalR 2 implementation works as expected. But after sometime (it can be after 5 minutes or 30 minutes or 10 minutes), the events are not called anymore. But the Server and Client are still connected. I know it is connected because it the trace level of the client for SignalR, it can still get the commands coming from the Server.
But my messages from my client is still being received by the server and its action executed so I know that the Server - Client connection is still good.
Also, I know that it is still connected because I always check the status of the connection.
This is how I implement the SignalR
public void ConnectAsync()
{
WriteToLogs("Connecting to Server...");
var dic = new Dictionary<string, string> { { "UserName", _signalrSettings.ApplicationName } };
_hubConnection?.Dispose();
_hubConnection = new HubConnection(_signalrSettings.ServerUri, dic);
_hubConnection.StateChanged += Connection_StateChanged;
_hubProxy = _hubConnection.CreateHubProxy(_signalrSettings.HubProxy);
try
{
_hubProxy.On<string, string>("Initial", (a, b) => Initial());
_hubConnection.Start().ContinueWith((s) =>
{
if (s.IsCompleted)
{
_isCompleted = true;
}
else if (s.IsCanceled)
{
_isCompleted = false;
}
else if (s.IsFaulted)
{
_isCompleted = false;
}
_hubProxy.On<string, string>("NewMessage", (userName, message) => NewMessage(userName, message)); // THE "NewMessage" is not fired after some time.
WriteToLogs( $#"Connection status is {_hubConnection.State}");
}).Wait();
}
catch (Exception e)
{
var trace = new StackTrace(e, true);
var reflectedType = trace.GetFrame(0).GetMethod().ReflectedType;
if (reflectedType != null)
{
_messageOperation.WriteErrors($#"Error {e.StackTrace} at {trace}");
}
}
}
I want to back up rocksdb while not blocking my writing calls, the status I expected is rocksdb should back up the data at that moment and ignoring the writing calls while backing up, is there a proper way to accomplish this?
I tested this in java using rocksdb-jni:
RocksDB.loadLibrary();
RocksDB rocksDb = RocksDB.open("C:/rocksdb/data");
BackupEngine engine = BackupEngine.open(rocksDb.getEnv(), new BackupEngineOptions("C:/rocksdb/backup"));
for(int i = 0;i < 100000;i++){
byte[] k = ByteBuffer.allocate(4).putInt(0, i).array();
rocksDb.put(k,k);
}
ExecutorService executorService = Executors.newFixedThreadPool(2);
CountDownLatch countDownLatch = new CountDownLatch(1);
executorService.execute(() -> {
try {
Thread.currentThread().setPriority(Thread.MAX_PRIORITY);
countDownLatch.await();
engine.createNewBackup(rocksDb, false);
} catch (RocksDBException | InterruptedException e) {
throw new RuntimeException(e);
}
});
executorService.execute(() -> {
try {
Thread.currentThread().setPriority(Thread.NORM_PRIORITY);
countDownLatch.await();
//Thread.sleep(1);
for(int i = 100000;i < 200000;i++){
byte[] k = ByteBuffer.allocate(4).putInt(0, i).array();
rocksDb.put(k,k);
}
} catch (InterruptedException | RocksDBException e) {
throw new RuntimeException(e);
}
});
countDownLatch.countDown();
the result I expected from restoring is exactly 100000 records, but I always got more records
Yes - both backup and checkpoint engines can achieve this i.e. take a live backup while not blocking writes
https://github.com/facebook/rocksdb/wiki/How-to-backup-RocksDB
I am receiving strings from my server that I want to append into a Textarea on the client side (Think chat window). Problem is, when I receive the string, the client freezes.
insertUserNameButton.setOnAction((event) -> {
userName=userNameField.getText();
try {
connect();
} catch (IOException e) {
e.printStackTrace();
}
});
public Client() {
userInput.setOnAction((event) -> {
out.println(userInput.getText());
userInput.setText("");
});
}
private void connect() throws IOException {
String serverAddress = hostName;
Socket socket = new Socket(serverAddress, portNumber);
in = new BufferedReader(new InputStreamReader(
socket.getInputStream()));
out = new PrintWriter(socket.getOutputStream(), true);
while (true) {
String line = in.readLine();
if (line.startsWith("SUBMITNAME")) {
out.println(userName);
} else if (line.startsWith("MESSAGE")) {
Platform.runLater(()->serverOutput.appendText(line.substring(8) + "\n"));
} else if (line.startsWith("QUESTION")) {
Platform.runLater(()->serverOutput.appendText(line.substring(8) + "\n"));
} else if (line.startsWith("CORRECTANSWER")) {
Platform.runLater(()->serverOutput.appendText(line.substring(14) + "\n"));
}
}
}
public static void main(String[] args) {
launch(args);
}
I have done some research and it seems that using Platform.runLater on each append should fix the problem. It doesn't for me.
Anyone has an idea of what it can be caused by? Thank you!
You are calling connect() on the FX Application Thread. Since it blocks indefinitely via the
while(true) {
String line = in.readLine();
// ...
}
construct, you block the FX Application Thread and prevent it from doing any of its usual work (rendering the UI, responding to user events, etc).
You need to run this on a background thread. It's best to use a Executor to manage the threads:
private final Executor exec = Executors.newCachedThreadPool(runnable -> {
Thread t = new Thread(runnable);
t.setDaemon(true);
return t ;
});
and then do
insertUserNameButton.setOnAction((event) -> {
userName=userNameField.getText();
exec.execute(() -> {
try {
connect();
} catch (IOException e) {
e.printStackTrace();
}
});
});
public class RiakSearch {
public static final String RIAK_SERVER = "10.11.172.17";
private static RiakCluster setUpCluster() throws UnknownHostException {
// This example will use only one node listening on localhost:10017
RiakNode node = new RiakNode.Builder()
.withRemoteAddress("10.11.172.17")
.withAuth("administrator", "password#123", null).build();
// This cluster object takes our one node as an argument
RiakCluster cluster = new RiakCluster.Builder(node).build();
// The cluster must be started to work, otherwise you will see errors
cluster.start();
return cluster;
}
public void uploadSchema() {
try {
RiakCluster cluster = setUpCluster();
RiakClient client = new RiakClient(cluster);
System.out.println("Client object successfully created");
File xml = new File("blog_post_schema.xml");
String xmlString = FileUtils.readFileToString(xml);
YokozunaSchema schema = new YokozunaSchema("blog_post_schema",
xmlString);
StoreSchema storeSchemaOp = new StoreSchema.Builder(schema).build();
client.execute(storeSchemaOp);
} catch (UnknownHostException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public static void main(String[] args) {
// TODO Auto-generated method stub
RiakSearch obj = new RiakSearch();
obj.uploadSchema();
}
}
java.util.concurrent.ExecutionException: com.basho.riak.client.core.netty.RiakResponseException: Unknown message code: 56
at com.basho.riak.client.core.FutureOperation.get(FutureOperation.java:260)
at com.basho.riak.client.api.commands.CoreFutureAdapter.get(CoreFutureAdapter.java:52)
at com.basho.riak.client.api.RiakCommand.execute(RiakCommand.java:89)
at com.basho.riak.client.api.RiakClient.execute(RiakClient.java:293)
at com.search.RiakSearch.main(RiakSearch.java:64)
Caused by: com.basho.riak.client.core.netty.RiakResponseException: Unknown message code: 56
at com.basho.riak.client.core.netty.RiakResponseHandler.channelRead(RiakResponseHandler.java:52)
at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:84)
at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:153)
at io.netty.channel.PausableChannelEventExecutor.invokeChannelRead(PausableChannelEventExecutor.java:86)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:389)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:243)
at io.netty.handler.codec.ByteToMessageCodec.channelRead(ByteToMessageCodec.java:103)
at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:84)
at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:153)
at io.netty.channel.PausableChannelEventExecutor.invokeChannelRead(PausableChannelEventExecutor.java:86)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:389)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:956)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:127)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:514)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:471)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:385)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:351)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at io.netty.util.internal.chmv8.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1412)
at io.netty.util.internal.chmv8.ForkJoinTask.doExec(ForkJoinTask.java:280)
at io.netty.util.internal.chmv8.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:877)
at io.netty.util.internal.chmv8.ForkJoinPool.scan(ForkJoinPool.java:1706)
at io.netty.util.internal.chmv8.ForkJoinPool.runWorker(ForkJoinPool.java:1661)
at io.netty.util.internal.chmv8.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:126)
Make sure that Solr is actually started. By default, search is disabled in Riak 2.x. In order to enable it, change search property in /etc/riak/riak.conf to on. Then restart Riak.
I had the similar issue of
RiakError: 'Unknown message code: 56'
I solved it by changing the parameter of search in the 'riak.conf' file
Here is the file location, if you are using mac and installed via brew
/usr/local/Cellar/riak/2.2.2/libexec/etc/riak.conf
Here are the lines of code i changed from off to on
## To enable Search set this 'on'.
##
## Default: off
##
## Acceptable values:
## - on or off
search = on
I found the documentation explanation a little bit tricky to follow but more or less it is the reference to solve the issue.
Can anybody help telling me what is wrong with my code? I am trying to connect to SQLite database, and executing some queries. when trying to create and open the database, create and insert the table, no exception returned. but when I try to execute delete statement,
DatabaseIOException: File system error (12)
always returned. I don't know the cause of the exception exactly. would you tell me what usually cause this kind of exception? I don't even know when I need to close the database and when I don't need to. this solution also makes me confused.
here is my code:
public class DatabaseManager {
Logger log = new Logger();
Database db;
public DatabaseManager() {
createDatabase();
}
private void createDatabase() {
// Determine if an SDCard is present
boolean sdCardPresent = false;
String root = null;
Enumeration enum = FileSystemRegistry.listRoots();
while (enum.hasMoreElements()) {
root = (String) enum.nextElement();
if(root.equalsIgnoreCase("sdcard/")) {
sdCardPresent = true;
}
}
if(!sdCardPresent) {
alert("This application requires an SD card to be present. Exiting application...");
}
else {
try {
URI uri = URI.create("/SDCard/databases/MyAdvanceUI/myadvanceui.db");
db = DatabaseFactory.openOrCreate(uri);
db.close();
//alert("Database OK!");
} catch (Exception e) {
// TODO Auto-generated catch block
//alert("Exception in createDatabase(): " + e);
}
}
}
private void alert(final String message) {
UiApplication.getUiApplication().invokeLater(new Runnable() {
public void run() {
Dialog.inform(message);
System.exit(0);
}
});
}
private void createTableTask() {
try {
URI uri = URI.create("/SDCard/databases/MyAdvanceUI/myadvanceui.db");
db = DatabaseFactory.open(uri);
Statement st = db.createStatement("CREATE TABLE IF NOT EXISTS t_task (id INTEGER PRIMARY KEY AUTOINCREMENT, "
+ "client TEXT, task TEXT)");
st.prepare();
st.execute();
st.close();
db.close();
//alert("Table Task created!");
} catch (Exception e) {
// TODO: handle exception
//alert("Exception in createTableTask(): " + e);
}
}
private void insertTableTask() {
String[] clients = { "Budi Setiawan", "Dian Kusuma", "Joko Ahmad", "Titi Haryanto", "Wahyu" };
String[] tasks = {
"Penawaran terhadap instalasi server",
"Follow up untuk keperluan produk terbaru",
"Pendekatan untuk membina relasi",
"Penawaran jasa maintenance",
"Penawaran terhadap instalasi database"
};
try {
URI uri = URI.create("/SDCard/databases/MyAdvanceUI/myadvanceui.db");
db = DatabaseFactory.open(uri);
for(int i = 0; i < clients.length; i++) {
Statement st = db.createStatement("INSERT INTO t_task (client, task) VALUES (?, ?)");
st.prepare();
st.bind(1, clients[i]);
st.bind(2, tasks[i]);
st.execute();
st.close();
}
db.close();
} catch (Exception e) {
// TODO: handle exception
//alert("Exception in insertTableTask(): " + e);
}
}
public void loadInitialData() {
createTableTask();
insertTableTask();
}
public Cursor getTasks() {
// TODO Auto-generated method stub
Cursor results = null;
try {
URI uri = URI.create("/SDCard/databases/MyAdvanceUI/myadvanceui.db");
db = DatabaseFactory.open(uri);
Statement st = db.createStatement("SELECT client, task FROM t_task");
st.prepare();
results = st.getCursor();
return results;
} catch (Exception e) {
// TODO: handle exception
//alert("Exception: " + e);
}
return results;
}
public void delete(String string) {
// TODO Auto-generated method stub
try {
URI uri = URI.create("/SDCard/databases/MyAdvanceUI/myadvanceui.db");
db = DatabaseFactory.open(uri);
Statement st = db.createStatement("DELETE FROM t_task WHERE client=?");
st.prepare();
st.bind(1, string);
st.execute();
} catch (Exception e) {
// TODO: handle exception
alert("Exception: " + e);
}
}
}
thank you for your help.
I don't see that you close the statement and close the database after select and delete actions. Most probably you can't open database because it wasn't closed correctly.
Big warning SD card isn't available when user mounted devices to PC as external drive. Some devices are going without SD card preinstalled. DB operations are really slow on 5 OS devices. Your alert method code wan't close db what could be issue to open it after on the next application start.
Warning As #pankar mentioned in comment you should add finally {} where you will close resources for sure. In your current implementation if you get exception in execution you will never close database.
Big improvements You don't need to create and prepare statement every loop. Just do it before for. Do bind and execute every loop. And close statement after for.
Improvements You could keep one opened db during application run cycle. It will save you some line of code and time for opening closing.
Notation It's bad practice to have parameter named like 'string'. I would rename it to something more meaningful.