RxJava-Aynchronous Stream Processing - asynchronous

Question

I am implementing a simple data analytic functionality with RXJava, where a topic subscriber asynchronously processes the data published to a topic, depositing the output to the Redis.
When a message is received, the Spring component publishes it to an Observable. To avoid blocking the submission I used the RxJava Async to do this asynchronously.
#Override
public void onMessage(final TransactionalMessage message) {
Async.start(new Func0<Void>() {
#Override
public Void call() {
analyser.process(message);
return null;
}
});
}
I have two confusions in implementing other processing parts; 1) creating an asynchronous Observable with buffering 2) Computing different logics in parallel based on message type on list of messages.
After long experiments I found two ways to create the Async Observable and not sure which one is the right and better approach.
Way one,
private static final class Analyzer {
private Subscriber<? super TransactionalMessage> subscriber;
public Analyzer() {
OnSubscribe<TransactionalMessage> f = subscriber -> this.subscriber = subscriber;
Observable.create(f).observeOn(Schedulers.computation())
.buffer(5, TimeUnit.SECONDS, 5, Schedulers.io())
.skipWhile((list) -> list == null || list.isEmpty())
.subscribe(t -> compute(t));
}
public void process(TransactionalMessage message) {
subscriber.onNext(message);
}
}
Way two
private static final class Analyser {
private PublishSubject<TransactionalMessage> subject;
public Analyser() {
subject = PublishSubject.create();
Observable<List<TransactionalMessage>> observable = subject
.buffer(5, TimeUnit.SECONDS, 5, Schedulers.io())
.observeOn(Schedulers.computation());
observable.subscribe(new Observer<List<TransactionalMessage>>() {
#Override
public void onCompleted() {
log.debug("[Analyser] onCompleted(), completed!");
}
#Override
public void onError(Throwable e) {
log.error("[Analyser] onError(), exception, ", e);
}
#Override
public void onNext(List<TransactionalMessage> t) {
compute(t);
}
});
}
public void process(TransactionalMessage message) {
subject.onNext(message);
}
}
The TransactionalMessage comes in different types, so I want to perform different computations based on the types. One approach I tried is filter the list based on every type and process them separately, but this looks so bad and I think does not work in parallel. What way to process them in parallel?
protected void compute(List<TransactionalMessage> messages) {
Observable<TransactionalMessage> observable = Observable
.from(messages);
Observable<String> observable2 = observable
.filter(new Func1<TransactionalMessage, Boolean>() {
#Override
public Boolean call(TransactionalMessage t) {
return t.getMsgType()
.equals(OttMessageType.click.name());
}
}).flatMap(
new Func1<TransactionalMessage, Observable<String>>() {
#Override
public Observable<String> call(
TransactionalMessage t) {
return Observable.just(
t.getMsgType() + t.getAppId());
}
});
Observable<String> observable3 = observable
.filter(new Func1<TransactionalMessage, Boolean>() {
#Override
public Boolean call(TransactionalMessage t) {
return t.getMsgType()
.equals(OttMessageType.image.name());
}
}).flatMap(
new Func1<TransactionalMessage, Observable<String>>() {
#Override
public Observable<String> call(
TransactionalMessage t) {
return Observable.just(
t.getMsgType() + t.getAppId());
}
});
// I sense some code smell in filtering on type and processing it.
Observable.merge(observable2, observable3)
.subscribe(new Action1<String>() {
#Override
public void call(String t) {
// save it to redis
System.out.println(t);
}
});
}

Answer 1

I suggest thinking about Subjects before attempting to use create.
If you want parallel processing done based on some categorization, you could use groupBy along with observeOn to achieve the desired effect:
Observable.range(1, 100)
.groupBy(v -> v % 3)
.flatMap(g ->
g.observeOn(Schedulers.computation())
.reduce(0, (a, b) -> a + b)
.map(v -> g.getKey() + ": " + v)
)
.toBlocking().forEach(System.out::println);