#Bean
public IntegrationFlow acknowledgeFileFlows() {
return f -> f
.enrichHeaders(h -> h
.headerExpression("originalPayload", "payload.toString()")
.headerExpression(FileHeaders.REMOTE_DIRECTORY, "payload.toString()"))
.log(LoggingHandler.Level.INFO, "eu.xxx", "'acknowledgeFileFlows Header originalPayload=' + headers[originalPayload]")
.handle(Sftp.outboundGateway(sessionFactory, Command.LS.getCommand(), "payload")
.autoCreateDirectory(false)
.autoCreateLocalDirectory(false)
.charset("UTF-8")
.regexFileNameFilter("(?i)(FileAcK_CITI_PMT[a-zA-Z0-9_.-]+|TxnAck_CITI_PMT[a-zA-Z0-9_.-]+|TxnNak_CITI_PMT[a-zA-Z0-9_.-]+)")
.options(Option.NAME_ONLY, Option.RECURSIVE))
.split()
.log(LoggingHandler.Level.INFO, "eu.xxx", "'acknowledgeFileFlows LS Payload= ' + payload.toString()")
.enrichHeaders(h -> h
.headerExpression("startTime", "new java.util.Date()")
.headerExpression("originalRemoteFile", "payload.toString()")
.headerExpression(FileHeaders.REMOTE_FILE, "payload.toString()"))
.handle(Sftp.outboundGateway(sessionFactory, Command.GET.getCommand(), "headers['originalPayload'] + headers['file_remoteFile']")
.autoCreateLocalDirectory(false)
.charset("UTF-8")
.fileExistsMode(FileExistsMode.REPLACE)
.fileNameExpression("headers['file_remoteFile']")
.localDirectoryExpression(new FunctionExpression<Message<?>>(m -> {
IntegrationMessageHeaderAccessor accessor = new IntegrationMessageHeaderAccessor(m);
final String remoteFileName = (String) accessor.getHeader("file_remoteFile");
final String ackFileRootPath = applicationConfiguration.getAcknowledgeFileTargetUri();
if (remoteFileName.toUpperCase().contains("XXX")) {
return Paths.get(ackFileRootPath, "XXX", RCIV, ACK).toString();
} else if (remoteFileName.toUpperCase().contains("YYY")) {
return Paths.get(ackFileRootPath, "YYY", RCIV, ACK).toString();
} else {
String[] parts = remoteFileName.toUpperCase().split("ZZZ");
if (parts != null && parts.length >= 2) {
return Paths.get(ackFileRootPath, "ZZZ", RCIV, ACK).toString();
}
}
return ackFileRootPath;
}))
.localFilenameExpression("headers['file_remoteFile']")
.options(Option.PRESERVE_TIMESTAMP)
.remoteFileSeparator("/"))
.handle((payload, headers) -> {
File file = (File) payload;
file.setWritable(true, false);
file.setReadable(true, false);
return file;
})
.enrichHeaders(h -> h
.headerExpression("fileLength", "payload.length")
.headerExpression("localDirectory", "payload.toString()"))
.wireTap(w -> w
.handle((payload, headers) -> {
return transactionLog("XXX_TR_ACK_FILE",
(Date) headers.get("startTime"),
(String) headers.get("localDirectory"),
(String) headers.get("file_remoteFile"),
headers.get("fileLength"));
})
.handle(Jpa.outboundAdapter(sourceIntegrationEntityManagerFactory)
.entityClass(TranComLog.class)
.persistMode(PersistMode.PERSIST), e -> e.transactional()))
.aggregate()
.channel(new NullChannel());
}
Above is the file download flows configuration.
This flow takes the remote SFTP path as the initial parameter.
LS -> File name process to get the correct target folder -> GET -> Write some log in to the DB.
The remote SFTP path can contain multiple files. I have to process those files one by one due to the requirement. (That's why I added the split)
My issue is when any exception occurs this flow stopped. I have to restart the server.
I think the reason is there is no correct exception handling in my flow, especially between the split and aggregate.
Does the Sftp.outboundGateway process the files one by one? If yes, the split and aggregate can be removed. Am I right?
If not, should I keep the split and aggregate? how can I add the proper exception handling?
Related
I am trying to use Reactive Kafka Receiver in order to process Kafka messages. I expected that consumer will give me new message only in case the last one I polled has been acknowledged. However, even if I do not run ack I get new records. Is it a desired behaviour or am I missing some configs?
void consumeMessages() {
kafkaConsumer(() -> true).subscribe();
}
Flux<Void> kafkaConsumer(BooleanSupplier repeatCondition) {
return reactiveConsumer
.receive()
.doOnError(error -> log.warn("Received error when get kafka message", error))
.doOnNext(receivedRecord -> log.debug("Received event {}", receivedRecord.value()))
.flatMap(this::handleReceivedRecord)
.onErrorResume(error -> Flux.empty())
.repeat(repeatCondition);
}
Flux<Void> handleReceivedRecord(ReceiverRecord<String, CustomEvent> receivedRecord) {
return Mono.just(receivedRecord)
.flatMapMany(
record ->
handleCustomEvent(record.value())
.doOnComplete(() -> {
log.debug("Committing offset for record: {}", record);
//record.receiverOffset().acknowledge();
})
.onErrorMap(error -> new ReceiverRecordException(record, error)))
.doOnError(error -> log.error("Error when processing CustomEvent", error))
.retryWhen(
Retry.backoff(consumerProperties.retries(), consumerProperties.minBackOff())
.transientErrors(true))
.onErrorResume(
error -> {
ReceiverRecordException ex = (ReceiverRecordException) error.getCause();
log.warn("Retries exhausted for " + ex.getRecord().value());
ex.getRecord().receiverOffset().acknowledge();
return Mono.empty();
});
}
I am trying to write a method that takes a username as its input and creates a SQL query statement which is then executed. As of right now, I am getting a NullPointer when I try to call result.getObject().
Here is my method for creating the SQL statement dynamically:
fun getByUsername(name: String): User?{
val sql = """SELECT * FROM usertable WHERE username="$name";"""
val result = DatabaseController().trySql(sql)
if (result != null) {
if (!result.isBeforeFirst) {
println("User was not found...")
return null
} else {
println("User found....")
return User(result.getObject("username").toString(),
result.getObject("password").toString(),
result.getObject("admin").toString().toBoolean())
}
}
return null
}
Method that executes query:
fun trySql(sqlCommand: String): ResultSet? {
var conn = this.connect()
println("trySql()-------Running query $sqlCommand")
var result = conn?.createStatement()?.executeQuery(sqlCommand)
conn?.close()
return result
}
The username I am running it on is in my table too, so that is not the issue. My method that creates a row in my table is working properly, so I know there is no issue with connecting to the database.
EDIT
If I just return result in trySql(), I no longer get any errors, but from my reading I am supposed to close connections after use?
Why you are having issues
You are closing the connection, which makes the ResultSet you get back no longer useful. You will have access to the result object, but the result object will be unable to read data from SQLite, leading to errors.
How to fix
You can pass some code to your trySql function, which will be performed with the connection still open, and then close after the passed block.
Example
Library
class Con(val con: Connection = DriverManager.getConnection(URL)): AutoCloseable by con {
fun <T> withSQL(query: String, block: (ResultSet)->T): T {
con.createStatement().use { statement ->
statement.executeQuery(query).use { results ->
return block(results)
}
}
}
}
Usage
Con().use { con ->
val result: Int = con.withSQL("SELECT * from example WHERE name='test'") {
it.getInt(1)
}
}
I need to get all image file from a particular directory, and all images in all recursive subdirectory for this directory.
You could use the Files API to get the image files:
Pattern imageMIMEPattern = Pattern.compile("image/.*");
button.setOnAction(evt -> {
DirectoryChooser chooser = new DirectoryChooser();
File f = chooser.showDialog(primaryStage);
if (f != null) {
Path p = f.toPath();
try {
// find all files with mime type image/... in subdirectories up to a depth of 10
Files.find(p, 10, (path, attributes) -> {
String contentType;
try {
contentType = Files.probeContentType(path);
} catch (IOException ex) {
return false;
}
return contentType != null && imageMIMEPattern.matcher(contentType).matches();
}).forEach(System.out::println);
} catch (IOException ex) {
throw new RuntimeException(ex);
}
}
});
Of course you need to replace .forEach(System.out::println) achieve the desired effect with the resulting Stream<Path> (e.g. map back to File/collect to List, ect.).
You also may want to use a different BiPredicate<Path, BasicFileAttributes> to determine the desired files. Furthermore you may want to search to a depth greater than 10.
If you want to follow links you need to add FileVisitOption.FOLLOW_LINKS as additional parameter to the Files.find method.
I've created an app using Swift3 and Xcode8, and using FMDB as my database, when it runs on the simulator it can get the data from data.db,but when it runs on the generic device (which is my phone), there's no data in the tableView, also could't insert records. I added data.db into my project, but when I changed records on simulator, records in data.db didn't change, but I printed the path, it pointed to simulator folder, database in that folder will changed along the modify in simulator and that folder's path changes almost every time. I'm so confused, is that because I didn't connected to my database in fact?
Here's the Utility.Swift which holds common and often reused function
import UIKit
class Utility: NSObject {
class func getPath(_ fileName: String) -> String {
let documentsURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0]
let fileURL = documentsURL.appendingPathComponent(fileName)
print(fileURL.path)
return fileURL.path
}
class func copyFile(_ fileName: NSString){
let dbPath: String = getPath(fileName as String)
let fileManager = FileManager.default
if !fileManager.fileExists(atPath: dbPath) {
let documentsURL = Bundle.main.resourceURL
let fromPath = documentsURL!.appendingPathComponent(fileName as String)
var error : NSError?
do {
try fileManager.copyItem(atPath: fromPath.path, toPath: dbPath)
}
catch let error1 as NSError {
error = error1
}
let alert: UIAlertView = UIAlertView()
if (error != nil) {
alert.title = "Error Occured"
alert.message = error?.localizedDescription
}
else {
alert.title = "Successfully Copy"
alert.message = "Your database copy successfully"
}
alert.delegate = nil
alert.addButton(withTitle: "Ok")
alert.show()
}
}
class func invokeAlertMethod(_ strTitle: NSString, strBody: NSString, delegate: AnyObject?) {
let alert: UIAlertView = UIAlertView()
alert.message = strBody as String
alert.title = strTitle as String
alert.delegate = delegate
alert.addButton(withTitle: "Ok")
alert.show()
}
}
and StudentDataBase.swift contains Query languages
import UIKit
let sharedInstance = StudentDataBase()
class StudentDataBase : NSObject {
var database: FMDatabase? = nil
class func getInstance() -> StudentDataBase{
if((sharedInstance.database) == nil)
{
sharedInstance.database = FMDatabase(path: Utility.getPath("data.db"))
}
return sharedInstance
}
func addStuData(_ student: Student) -> Bool{
sharedInstance.database!.open()
let isInserted = sharedInstance.database!.executeUpdate("INSERT INTO [Student info] (StudentID, FirstName, LastName, PhoneNumber) VALUES (?, ?, ?, ?)", withArgumentsIn: [student.studentID, student.fstName, student.lstName, student.phoneNum])
sharedInstance.database!.close()
return isInserted
}
func updateStuData(_ student: Student) -> Bool {
sharedInstance.database!.open()
let isUpdated = sharedInstance.database!.executeUpdate("UPDATE [Student info] SET FirstName=?, LastName=?, PhoneNumber=? WHERE StudentID=?", withArgumentsIn: [student.fstName, student.lstName, student.phoneNum, student.studentID])
print(student)
print(isUpdated)
sharedInstance.database!.close()
return isUpdated
}
func deleteStuData(_ student: Student) -> Bool {
sharedInstance.database!.open()
let isDeleted = sharedInstance.database!.executeUpdate("DELETE FROM [Student info] WHERE StudentID=?", withArgumentsIn: [student.studentID])
sharedInstance.database!.close()
return isDeleted
}
func getAllStuData() -> [Student] {
sharedInstance.database!.open()
let resultSet: FMResultSet! = sharedInstance.database!.executeQuery("SELECT * FROM [Student info]", withArgumentsIn: nil)
var marrStudentInfo : [Student] = []
if (resultSet != nil) {
while resultSet.next() {
let student : Student = Student()
student.studentID = resultSet.string(forColumn: "StudentID")
student.fstName = resultSet.string(forColumn: "FirstName")
student.lstName = resultSet.string(forColumn: "LastName")
student.phoneNum = resultSet.string(forColumn: "PhoneNumber")
marrStudentInfo.append(student)
}
}
sharedInstance.database!.close()
return marrStudentInfo
}
}
also in AppDelegate.swift, I've written:
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {
// Override point for customization after application launch.
Utility.copyFile("data.db")
return true
}
I'm new to Swift and FMDB, please explain and I'll try my best.Thank you very much!!!!
Edit:
After I downloaded the contents in my phone and the database is blank. And after I throwaway the error, it shows
Inserted failed:Optional("no such table: Student info")
SQLite offers good error reporting and you should avail yourself of it. So, for example, if any of these executeUpdate calls fail, you should examine the lastErrorMessage (before you close the database) so you know why it failed.
Personally, I'd suggest using executeQuery(_:values:) and executeUpdate(_:values:) rather than executeQuery(_:withArgumentsIn:) and executeUpdate(_:withArgumentsIn:) because the former methods throw errors (which shifts to a more active error handling paradigm from the more passive one in which you have to manually remember to check for errors). But regardless of how you do your error handling, do it, or else we're all guessing.
Personally, I'd suggest using Xcode's "Devices" window and download your app's bundle (see https://stackoverflow.com/a/38064225/1271826) and look at the database. I bet it's a blank database with no tables defined at all. In terms of why, it could be because you did some earlier testing and there's an old version of the database in the Documents folder. Try completely deleting the app from the device and re-running and see if that fixes it.
Other, more obscure sources of problem include filename capitalization (e.g. Data.db vs data.db). The macOS file system will often handle that gracefully because it's not (generally) case sensitive. But iOS devices are case sensitive.
But, like I said, we're flying blind until (a) you add some error handling to your code so you can see why its failing; and (b) look at the database on the device to see what it looks like.
I made a Javafx scene to handle drag-n-drop and it is working fine if you drag a file from the windows explorer or from the desktop for example.
However, if I try to do it from Outlook, the behavior is weird.
I realized that when you drag n drop from another program inside the dragboard component, the method "getContentTypes" will return a few DataFormat objects using this code:
dragField.setOnDragOver((DragEvent event) -> {
Dragboard db = event.getDragboard();
System.out.println(db.getContentTypes());
});
The output will be something like:
[text/plain] - De Objet Reçu Taille Catégories D D Test 13:56 40 Ko [DragImageBits] -
java.nio.HeapByteBuffer[pos=0 lim=90304 cap=90304]
[message/external-body;access-type=clipboard;index=0;name="testEmail.msg"] -
null [Object Descriptor] - java.nio.HeapByteBuffer[pos=0 lim=74
cap=74] [RenPrivateItem] - null [CSV] - java.nio.HeapByteBuffer[pos=0
lim=282 cap=282]
It seems to be able to extract the information from the Outlook msg file, since I got something like a header and also the "testEmail.msg" file name is correct.
However, when I try to use this code:
DataFormat.lookupMimeType("message/external-body;access-type=clipboard;index=0;name=\"testEmail.msg\"");
It returns null... In fact, there is a "null" by the mime-type side.
Is there any way to transform these DataFormat objects into a java file or maybe a apache poi msg file? Anything would be amazing.
Thanks for any help! :D
If there is a mime type starting with message/external-body;access-type=clipboard, you can get the value using
clipboard.getContent(new DataFormat("message/external-body"));
Here is an example which just saves the file:
private boolean clipboardHasInMemoryFile(Clipboard clipboard) {
for (DataFormat d: clipboard.getContentTypes()) {
if (d.toString().startsWith("[message/external-body;access-type=clipboard")) {
return true;
}
}
return false;
}
public void saveOutlookFile() throws IOException {
Clipboard clipboard = Clipboard.getSystemClipboard();
if (clipboardHasInMemoryFile(clipboard)) {
//this is for copying an outlook attachment
String name = "outfile";
for (DataFormat d : clipboard.getContentTypes()) {
if (d.toString().startsWith("[message/external-body;access-type=clipboard")) {
Pattern p = Pattern.compile("name=\"([^\"]*)\"");
Matcher m = p.matcher(d.toString());
m.find();
name = m.group(1);
break;
}
}
Object inMemoryFile = null;
try {
DataFormat df = DataFormat.lookupMimeType("message/external-body");
if (df == null) {
df = new DataFormat("message/external-body");
}
inMemoryFile = clipboard.getContent(df);
} catch (Throwable t) {
}
final String fileName = name;
if (inMemoryFile != null) {
if (inMemoryFile instanceof ByteBuffer) {
ByteBuffer b = (ByteBuffer) inMemoryFile;
byte bytes[] = b.array();
FileOutputStream fo = new FileOutputStream(fileName);
fo.write(b.array());
fo.close();
}
}
}
}