rxjava - Combine onerror and timout handling - asynchronous

I will start with what I want to achieve.
I want to call a method that returns an Observabe.
I do not know if the called method handles exceptions and timeouts
I want to combine observables in my call (merge/zip etc)
if one method fails, I want the answers from the methods that succeeded -
I don't want to break the flow.
In case of exception, I am capable of handling it and continuing with the flow,
but when I try to add timeoutmanagement I fail.
Here is my code
public static void main(String[] args) {
createObservables(false, true); // stalls for timeout
zip();
}
private static void createObservables(final boolean throwException,
final boolean stall) {
obs1 = Observable.just(1);
obs1 = obs1.map(new Func1<Integer, Integer>() {
#Override public Integer call(Integer integer) {
int i = 0;
if (throwException)
getObj().equals("");
if (stall)
zzz(10);
return ++integer;
}
});
obs2 = Observable.just(111);
}
private static void zip() {
System.out.println("**Zip**");
obs1 = obs1.onErrorReturn(new Func1<Throwable, Integer>() {
#Override public Integer call(Throwable throwable) {
return 999;
}
});
obs1 = obs1.timeout(5, TimeUnit.SECONDS);
Observable.zip(obs1, obs2, new Func2<Integer, Integer, ArrayList<Integer>>() {
#Override
public ArrayList<Integer> call(Integer integer1, Integer integer2) {
ArrayList<Integer> integers = new ArrayList<Integer>();
integers.add(integer1);
integers.add(integer2);
return integers;
}
}).subscribe(new Observer<Object>() {....}
);
}
Now, when I call
createObservables(false , false); // no exceptions and timeouts
I get onNext - [2, 111].
then I call
createObservables(true, false); // throw exception in one method only
I get onNext - [999, 111] - which is what I want. Exception and the result from the second method.
But when I call
createObservables(false, true); // stall on timeout
I get only onError.
But I want to get the other method answer.
Thanks.

Try creating an observable for your timeout value, in this case you want the same value as your error case:
Observable obs1Timeout = Observable.just(999);
Then in your timeout policy give it this observable as the fallback to use in the case of a timeout:
obs1 = obs1.timeout(5, TimeUnit.SECONDS, obs1Timeout);

Related

Multiple chained API calls to fetch data, but doOnNext of PublishSubject is never reached

I have a problem to understand a chained "RXJava-Retrofit" API call. I got inspired by this and implement this class named ObservationLoader to load the data from the API bucket per bucket. When the end of data is reached the API sends a endOfRecords=true:
public Observable<PageObject<Observation>> getAllObservationDataByRegion(long taxonKey,
String regionId) {
final PublishSubject<PageObject<Observation>> subject = PublishSubject.create();
return subject.doOnSubscribe(disposable -> {
this.getData(taxonKey, regionId, 0).subscribe(subject);
})
.doOnNext(observationPageObject -> {
if (observationPageObject.isEndOfRecords()) {
// -> list is completely loaded
subject.onComplete();
} else {
int nextOffset = observationPageObject.getOffset() + 1;
this.getData(taxonKey, regionId, null, nextOffset).subscribe(subject);
}
})
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread());
}
private Observable<PageObject<Observation>> getData(long id,
String regionId,
int offset) {
// Get your API response value
return this.api.getObservations(id, regionId, ObservationLoader.PAGE_LIMIT, offset);
}
In my Android fragment HomeFragment I subscribe to the ObservationLoader:
ObservationLoader loader = new ObservationLoader(this.getApi());
Observable<PageObject<Observation>> observable = loader
.getAllObservationDataByRegion(this.getSelectedSpecies(), this.getSelectedRegion());
observable.subscribe(new Observer<PageObject<Observation>>() {
#Override
public void onSubscribe(Disposable d) {
Log.i(TAG, "ON_SUBSCRIBE");
}
#Override
public void onNext(PageObject<Observation> observationPageObject) {
Log.i(TAG, "ON_NEXT");
}
#Override
public void onError(Throwable e) {
Log.i(TAG, "ERROR = " + e.getMessage());
}
#Override
public void onComplete() {
Log.i(TAG, "COMPLETED");
}
});
I can see that the onSubscribe() and doOnSubscribe() are called and even the getData() is reached. I assume the API is responding correctly (a previous attempt attempt with recursion worked fine). But I never reached the doOnNext function. The observer goes straight to onComplete() and no data is received. What could be the reason?
When doOnSubscribe runs, the doesn't see any consumers yet so if getData is synchronous, there won't be any first results to trigger further results. Also if getData ends, it will complete the setup so the next getData call in doOnNext will push to an already terminated subject, ingoring all data.
You'll need a differently organized feedback loop:
// we loop back the nextOffset, in a thread-safe manner
Subject<Integer> subject = PublishSubject.<Integer>create()
.toSerialized();
// bootstrap with 0 and keep open for more offsets
subject.mergeWith(Observable.just(0))
// get the data for the current offset
.concatMap(nextOffset -> getData(taxonKey, regionId, nextOffset)
.subscribeOn(Schedulers.io())
)
// if the response is end of records, stop
.takeWhile(observationPageObject -> !observationPageObject.isEndOfRecords())
// otherwise not end of records, feedback the new offset
.doOnNext(observationPageObject ->
subject.onNext(observationPageObject.getOffset() + 1)
)
// get the data on the main thread
.observeOn(AndroidSchedulers.mainThread());

spring-kafka kafkaStreamsBuilder.getKafkaStreams() is null

Here is my code
The first bean is watching the messages on Topic.TRANSACTION_RAW and split one message into two and send them to Topic.TRANSACTION_INTERNAL
And the second bean is doing group and reducing and materialize it to the state store "StateStore.BALANCE".
The last one is to get the ReadOnlyKeyValueStore to read state from "ReadOnlyKeyValueStore".
#Configuration(proxyBeanMethods = false)
#EnableKafkaStreams
public class MyKafkaStreamsConfiguration {
#Bean
public KStream<String, BankTransaction> alphaBankKStream(StreamsBuilder streamsBuilder) {
JsonSerde<BankTransaction> valueSerde = new JsonSerde<>(BankTransaction.class);
KStream<String, BankTransaction> stream = streamsBuilder.stream(Topic.TRANSACTION_RAW,
Consumed.with(Serdes.String(), valueSerde));
stream.flatMap((k, v) -> {
List<BankTransactionInternal> txInternals = BankTransactionInternal.splitBankTransaction(v);
List<KeyValue<String, BankTransactionInternal>> result = new LinkedList<>();
result.add(KeyValue.pair(v.getFromAccount(), txInternals.get(0)));
result.add(KeyValue.pair(v.getToAccount(), txInternals.get(1)));
return result;
}).filter((k, v) -> !Constants.EXTERNAL_ACCOUNT.equalsIgnoreCase(k))
.to(Topic.TRANSACTION_INTERNAL, Produced.with(Serdes.String(), new JsonSerde<>()));
return stream;
}
#Bean
public KStream<String, BankTransactionInternal> alphaBankInternalKStream(StreamsBuilder streamsBuilder) {
JsonSerde<BankTransactionInternal> valueSerde = new JsonSerde<>(BankTransactionInternal.class);
KStream<String, BankTransactionInternal> stream = streamsBuilder.stream(Topic.TRANSACTION_INTERNAL,
Consumed.with(Serdes.String(), valueSerde));
KGroupedStream<String, Double> groupedByAccount = stream
.map((k,v) -> KeyValue.pair(k, v.getAmount()))
.groupBy((account, amount) -> account, Grouped.with(Serdes.String(), Serdes.Double()));
groupedByAccount.reduce(Double::sum,
Materialized.<String, Double, KeyValueStore<Bytes, byte[]>>as(StateStore.BALANCE)
.withValueSerde(Serdes.Double()));
return stream;
}
#Bean
public ReadOnlyKeyValueStore<String, Double> balanceStateStore(StreamsBuilderFactoryBean defaultKafkaStreamsBuilder) {
if (defaultKafkaStreamsBuilder == null) {
System.out.println("... defaultKafkaStreamsBuilder is null ...");
}
if (defaultKafkaStreamsBuilder.getKafkaStreams() == null) {
System.out.println("... defaultKafkaStreamsBuilder.getKafkaStreams() is null ...");
// this one got printed
}
ReadOnlyKeyValueStore<String, Double> store = defaultKafkaStreamsBuilder.getKafkaStreams().store(
StateStore.BALANCE,
QueryableStoreTypes.keyValueStore());
return store;
}
}
I always got NullPointException on defaultKafkaStreamsBuilder.getKafkaStreams().
Any idea what is wrong here? Thanks!
if (defaultKafkaStreamsBuilder.getKafkaStreams() == null) {
System.out.println("... defaultKafkaStreamsBuilder.getKafkaStreams() is null ...");
// this one got printed
}
This operation is not good to do during bean definition phase.
See its JavaDocs:
/**
* Get a managed by this {#link StreamsBuilderFactoryBean} {#link KafkaStreams} instance.
* #return KafkaStreams managed instance;
* may be null if this {#link StreamsBuilderFactoryBean} hasn't been started.
* #since 1.1.4
*/
public synchronized KafkaStreams getKafkaStreams() {
since you call this method far too early before a lifecycle start phase, you end-up with that error.
You should reconsider your logic in favor of SmartLifecycle.start() in the target service where you'd like to use that ReadOnlyKeyValueStore. So, you autowire over there this StreamsBuilderFactoryBean and call its getKafkaStreams() from the start() implementation.

WebFlux returning http.okay vice http.notFound

New to WebFlux, reactive, and handlers. I've got things "working", but am not understanding why following code is returning "okay" with empty body, vice "not found".
Clarification: The issue-of-concern is in the final return statement of DemoPOJOHandler.getById(). The "short-circuit" code works as expected (i.e., returns "Bad Request" status), but the "switchIfEmpty" path of the final return statement does not appear to get exercised if a DemoPOJORepo.getById(int) returns Mono.empty().
(Note: I've hacked up a list-based "repo" to avoid dealing with database while figuring out handlers and http return types.)
Router implementation ("/v1" is a set of annotation based RESTful endpoints)...
#Configuration
public class DemoPOJORouter {
#Bean
public RouterFunction<ServerResponse> route(DemoPOJOHandler requestHandler) {
return nest(path("/v2"),
nest(accept(APPLICATION_JSON),
RouterFunctions.route(RequestPredicates.GET("/DemoPOJO"), requestHandler::getAll)
.andRoute(RequestPredicates.GET("/DemoPOJO/{id}"), requestHandler::getById)
.andRoute(RequestPredicates.POST("/DemoPOJO"), requestHandler::add)));
}
}
Handler implementation has been "stripped down" to only the code in question. I have a feeling that much of the style is "still imperative", but I've attempted to put the reactive stuff where it "makes the most sense".
If I supply a bad value on the URI (i.e., "foo"), then I get the http "bad request" returned. But, never seem to get the "not found" that should be generated by "switchIfEmpty" if a validly formatted int value is supplied, but it does not map to an entry in the repo.
#Component
public class DemoPOJOHandler {
public static final String PATH_VAR_ID = "id";
private DemoPOJORepo repo = null;
public Mono<ServerResponse> getById(ServerRequest request) {
Mono<DemoPOJO> monoDemoPOJO = null;
Map<String, String> pathVariables = request.pathVariables();
int id = -1;
checkRepoRef(); // part of the list hack
// short-circuit if request doesn't contain id (should never happen)
if ((pathVariables == null)
|| (!pathVariables.containsKey(PATH_VAR_ID))) {
return ServerResponse.badRequest().build();
}
// short-circuit if bad id value
try {
id = Integer.parseInt(pathVariables.get(PATH_VAR_ID));
} catch(NumberFormatException e) {
return ServerResponse.badRequest().build();
}
// get entity by keyValue
monoDemoPOJO = repo.getById(id);
return monoDemoPOJO
.flatMap(demoPOJO -> ServerResponse.ok()
.contentType(MediaType.APPLICATION_JSON)
.syncBody(demoPOJO)
.switchIfEmpty(ServerResponse.notFound().build()));
}
}
Hack of a list-based repo to avoid dealing with data/APIs while working on handlers and http return types.
// local hack to avoid a database for testing
public class DemoPOJORepo {
private static DemoPOJORepo fpRepo = null;
private static int NUM_ROWS = 100;
private Map<Integer, DemoPOJO> fooPOJOMap;
private DemoPOJORepo() {
initMap();
}
public static DemoPOJORepo getInstance() {
if (fpRepo == null) {
fpRepo = new DemoPOJORepo();
}
return fpRepo;
}
public Mono<DemoPOJO> getById(int id) {
Mono<DemoPOJO> monoDP;
if (fooPOJOMap.containsKey(id)) {
monoDP = Mono.just(fooPOJOMap.get(id));
} else {
monoDP = Mono.empty();
}
return monoDP;
}
private Mono<Void> initMap() {
fooPOJOMap = new TreeMap<Integer, DemoPOJO>();
int offset = -1;
for(int ndx=0; ndx<NUM_ROWS; ndx++) {
offset = ndx + 1;
fooPOJOMap.put(offset, new DemoPOJO(offset, "foo_" + offset, offset+100));
}
return Mono.empty();
}
}
Your brackets are in the wrong place causing the swithIfEmpy to apply to the ServerResponse.ok() publisher not the monoDemoPOJO, replace the return with this and it should work:
return monoDemoPOJO
.flatMap(demoPOJO -> ServerResponse.ok().contentType(MediaType.APPLICATION_JSON).syncBody(demoPOJO))
.switchIfEmpty(ServerResponse.notFound().build());
As I can see the code is right. The response code is Bad request because you are trying to convert "foo" to Integer, and when it throws an exception you are returning a Bad request response, so I think it works perfectly fine.
If you use an Integer id that is not present in your database then the answer must be a not found response

toCompletableFuture() stucks for asynchronous cache

Hello there I am trying to force a promise to end to get the result from it but it just stucks on loading.
public class CacheController extends Controller {
private AsyncCacheApi cache;
public Result cache()
{
String test = "nice";
cache.set("item.key", test, 15);
Customer user = new Customer("Ana", 12);
CompletionStage<Done> result = cache.set(user.getName(), user);
block(result);
return ok("Cached");
}
public Result checkCache() throws Exception
{
Logger.info("start");
//CompletionStage<String> news = cache.get("item.key");
//news.thenRun(() -> System.out.println("works"));
CompletionStage<Customer> result = cache.get("Ana");
Logger.info("step 1");
Logger.info(cache.get("Ana").toString());
Logger.info("Step 2");
Customer c = block(result);
Logger.info("Step 3 " + c.getName());
//result.thenRun(() -> setUser(result)).thenRun(() -> Logger.info(user.getName() + " " + user.getAge()));
return ok("cancan");
}
private <T> T block(CompletionStage<T> stage) {
try {
return stage.toCompletableFuture().get();
} catch (Throwable e) {
throw new RuntimeException(e);
}
}
}
When trying to load the page it gets stuck after step2 at line 56: Customer c = block(result); by my guesses
Any ideas to fix it?
#Codrin
I had the same problem. But, see https://www.playframework.com/documentation/2.6.x/JavaCache#Setting-the-execution-context
By default, all Ehcache operations are blocking, and async implementations will block threads in the default execution context.
Maybe CompletableFuture.get() gets stuck because it is executed in the same thread with the caller.
Referring to the linked page, I added snippet below to my application.conf and it worked.
play.cache.dispatcher = "contexts.blockingCacheDispatcher"
contexts {
blockingCacheDispatcher {
fork-join-executor {
parallelism-factor = 3.0
}
}
}

AspectJ - Is is possible to extend an enum's value?

Say I have an enum
public enum E {A,B,C}
Is it possible to add another value, say D, by AspectJ?
After googling around, it seems that there used to be a way to hack the private static field $VALUES, then call the constructor(String, int) by reflection, but seems not working with 1.7 anymore.
Here are several links:
http://www.javaspecialists.eu/archive/Issue161.html (provided by #WimDeblauwe )
and this: http://www.jroller.com/VelkaVrana/entry/modify_enum_with_reflection
Actually, I recommend you to refactor the source code, maybe adding a collection of valid region IDs to each enumeration value. This should be straightforward enough for subsequent merging if you use Git and not some old-school SCM tool like SVN.
Maybe it would even make sense to use a dynamic data structure altogether instead of an enum if it is clear that in the future the list of commands is dynamic. But that should go into the upstream code base. I am sure the devs will accept a good patch or pull request if prepared cleanly.
Remember: Trying to avoid refactoring is usually a bad smell, a symptom of an illness, not a solution. I prefer solutions to symptomatic workarounds. Clean code rules and software craftsmanship attitude demand that.
Having said the above, now here is what you can do. It should work under JDK 7/8 and I found it on Jérôme Kehrli's blog (please be sure to add the bugfix mentioned in one of the comments below the article).
Enum extender utility:
package de.scrum_master.util;
import java.lang.reflect.AccessibleObject;
import java.lang.reflect.Array;
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import sun.reflect.ConstructorAccessor;
import sun.reflect.FieldAccessor;
import sun.reflect.ReflectionFactory;
public class DynamicEnumExtender {
private static ReflectionFactory reflectionFactory =
ReflectionFactory.getReflectionFactory();
private static void setFailsafeFieldValue(Field field, Object target, Object value)
throws NoSuchFieldException, IllegalAccessException
{
// let's make the field accessible
field.setAccessible(true);
// next we change the modifier in the Field instance to
// not be final anymore, thus tricking reflection into
// letting us modify the static final field
Field modifiersField = Field.class.getDeclaredField("modifiers");
modifiersField.setAccessible(true);
int modifiers = modifiersField.getInt(field);
// blank out the final bit in the modifiers int
modifiers &= ~Modifier.FINAL;
modifiersField.setInt(field, modifiers);
FieldAccessor fa = reflectionFactory.newFieldAccessor(field, false);
fa.set(target, value);
}
private static void blankField(Class<?> enumClass, String fieldName)
throws NoSuchFieldException, IllegalAccessException
{
for (Field field : Class.class.getDeclaredFields()) {
if (field.getName().contains(fieldName)) {
AccessibleObject.setAccessible(new Field[] { field }, true);
setFailsafeFieldValue(field, enumClass, null);
break;
}
}
}
private static void cleanEnumCache(Class<?> enumClass)
throws NoSuchFieldException, IllegalAccessException
{
blankField(enumClass, "enumConstantDirectory"); // Sun (Oracle?!?) JDK 1.5/6
blankField(enumClass, "enumConstants"); // IBM JDK
}
private static ConstructorAccessor getConstructorAccessor(Class<?> enumClass, Class<?>[] additionalParameterTypes)
throws NoSuchMethodException
{
Class<?>[] parameterTypes = new Class[additionalParameterTypes.length + 2];
parameterTypes[0] = String.class;
parameterTypes[1] = int.class;
System.arraycopy(additionalParameterTypes, 0, parameterTypes, 2, additionalParameterTypes.length);
return reflectionFactory.newConstructorAccessor(enumClass .getDeclaredConstructor(parameterTypes));
}
private static Object makeEnum(Class<?> enumClass, String value, int ordinal, Class<?>[] additionalTypes, Object[] additionalValues)
throws Exception
{
Object[] parms = new Object[additionalValues.length + 2];
parms[0] = value;
parms[1] = Integer.valueOf(ordinal);
System.arraycopy(additionalValues, 0, parms, 2, additionalValues.length);
return enumClass.cast(getConstructorAccessor(enumClass, additionalTypes).newInstance(parms));
}
/**
* Add an enum instance to the enum class given as argument
*
* #param <T> the type of the enum (implicit)
* #param enumType the class of the enum to be modified
* #param enumName the name of the new enum instance to be added to the class
*/
#SuppressWarnings("unchecked")
public static <T extends Enum<?>> void addEnum(Class<T> enumType, String enumName) {
// 0. Sanity checks
if (!Enum.class.isAssignableFrom(enumType))
throw new RuntimeException("class " + enumType + " is not an instance of Enum");
// 1. Lookup "$VALUES" holder in enum class and get previous enum
// instances
Field valuesField = null;
Field[] fields = enumType.getDeclaredFields();
for (Field field : fields) {
if (field.getName().contains("$VALUES")) {
valuesField = field;
break;
}
}
AccessibleObject.setAccessible(new Field[] { valuesField }, true);
try {
// 2. Copy it
T[] previousValues = (T[]) valuesField.get(enumType);
List<T> values = new ArrayList<T>(Arrays.asList(previousValues));
// 3. build new enum
T newValue = (T) makeEnum(
enumType, // The target enum class
enumName, // THE NEW ENUM INSTANCE TO BE DYNAMICALLY ADDED
values.size(), new Class<?>[] {}, // could be used to pass values to the enum constuctor if needed
new Object[] {} // could be used to pass values to the enum constuctor if needed
);
// 4. add new value
values.add(newValue);
// 5. Set new values field
setFailsafeFieldValue(valuesField, null, values.toArray((T[]) Array.newInstance(enumType, 0)));
// 6. Clean enum cache
cleanEnumCache(enumType);
} catch (Exception e) {
e.printStackTrace();
throw new RuntimeException(e.getMessage(), e);
}
}
}
Sample application & enum:
package de.scrum_master.app;
/** In honour of "The Secret of Monkey Island"... ;-) */
public enum Command {
OPEN, CLOSE, PUSH, PULL, WALK_TO, PICK_UP, TALK_TO, GIVE, USE, LOOK_AT, TURN_ON, TURN_OFF
}
package de.scrum_master.app;
public class Server {
public void executeCommand(Command command) {
System.out.println("Executing command " + command);
}
}
package de.scrum_master.app;
public class Client {
private Server server;
public Client(Server server) {
this.server = server;
}
public void issueCommand(String command) {
server.executeCommand(
Command.valueOf(
command.toUpperCase().replace(' ', '_')
)
);
}
public static void main(String[] args) {
Client client = new Client(new Server());
client.issueCommand("use");
client.issueCommand("walk to");
client.issueCommand("undress");
client.issueCommand("sleep");
}
}
Console output with original enum:
Executing command USE
Executing command WALK_TO
Exception in thread "main" java.lang.IllegalArgumentException: No enum constant de.scrum_master.app.Command.UNDRESS
at java.lang.Enum.valueOf(Enum.java:236)
at de.scrum_master.app.Command.valueOf(Command.java:1)
at de.scrum_master.app.Client.issueCommand(Client.java:12)
at de.scrum_master.app.Client.main(Client.java:22)
Now you can either add an aspect with an advice executed after the enum class was loaded or just call this manually in your application before extended enum values are to be used for the first time. Here I am showing how it can be done in an aspect.
Enum extender aspect:
package de.scrum_master.aspect;
import de.scrum_master.app.Command;
import de.scrum_master.util.DynamicEnumExtender;
public aspect CommandExtender {
after() : staticinitialization(Command) {
System.out.println(thisJoinPoint);
DynamicEnumExtender.addEnum(Command.class, "UNDRESS");
DynamicEnumExtender.addEnum(Command.class, "SLEEP");
DynamicEnumExtender.addEnum(Command.class, "WAKE_UP");
DynamicEnumExtender.addEnum(Command.class, "DRESS");
}
}
Console output with extended enum:
staticinitialization(de.scrum_master.app.Command.<clinit>)
Executing command USE
Executing command WALK_TO
Executing command UNDRESS
Executing command SLEEP
Et voilà! ;-)

Resources