Years ago on a different project, we were able to import easily large xml property files with all the custom queries we needed that could not be generated, safely wrapped in CDATA tags. Have not found the similar option in Spring Boot.
Here is what we currently have:
#Value("classpath:sql/query.sql")
private Resource queryFile;
private String query;
#PostConstruct
public void loadNewSQL() throws IOException {
InputStream is = queryFile.getInputStream();
this.query = org.apache.commons.io.IOUtils.toString(is);
}
Previously, it was config for one file and all the content were accessible by the #Value annotation. Still cannot find the modern equivalent.
Related
I'm trying to implement an RSS/Atom feed aggregator in spring-integration and I am primarily using the Java DSL to write my IntegrationFlow. A requirement of this aggregator is that feeds can be added / removed during runtime. That is to say, the feeds are not known at design time.
I found it simple to use the basic Feed.inboundAdapter() with a test url and extract the links out of the feed with a transformer and then pass it on to an outbound-file-adapter to save the links to a file. However, I have gotten very stuck when trying to read the (thousands) of feed urls from an inbound-file-adapter run the file through a FileSplitter and then pass each resulting Message<String> containing the feed url to then register a new Feed.inboundAdapter(). Is this not possible with the Java DSL?
Ideally I would love it if I could do the following:
#Bean
public IntegrationFlow getFeedsFromFile() throws MalformedURLException {
return IntegrationFlows.from(inboundFileChannel(), e -> e.poller(Pollers.fixedDelay(10000)))
.handle(new FileSplitter())
//register new Feed.inboundAdapter(payload.toString()) foreach Message<String> containing feed url coming from FileSplitter
.transform(extractLinkFromFeedEntry())
.handle(appendLinkToFile())
.get();
}
Though after reading through the spring integration java DSL code multiple times (and learning a tonne of stuff along the way) I just can't see that it's possible to do it this way. So... A) is it? B) should it be? C) Suggestions?
It almost feels like I should be able to take the output of .handle(new FileSplitter()) and pass that into .handleWithAdapter(Feed.inboundAdapter(/*stuff here*/)) but the DSL only references outbound-adapters there. Inbound adapters are really just a subclass of AbstractMessageSource and it seems the only place you can specify one of those is as an argument to the IntegrationFlows.from(/*stuff here*/) method.
I would have thought it would be possible to take the input from a file, split it line by line, use that output to register inbound feed adapters, poll those feeds, extract the new links from feeds as they appear and append them to a file. It appears as though it's not.
Is there some clever subclassing I can do to make this work??
Failing that... and I suspect this is going to be the answer, I found the spring integration Dynamic Ftp Channel Resolver Example and this answer on how to adapt it dynamically register stuff for the inbound case...
So is this the way to go? Any help/guidance appreciated. After pouring over the DSL code and reading documentation for days, I think I'll have a go at implementing the dynamic ftp example and adapting it to work with FeedEntryMessageSource... in which case my question is... that dynamic ftp example works with XML configuration, but is it possible to do it with either Java config or the Java DSL?
Update
I've implemented the solution as follows:
#SpringBootApplication
class MonsterFeedApplication {
public static void main(String[] args) throws IOException {
ConfigurableApplicationContext parent = SpringApplication.run(MonsterFeedApplication.class, args);
parent.setId("parent");
String[] feedUrls = {
"https://1nichi.wordpress.com/feed/",
"http://jcmuofficialblog.com/feed/"};
List<ConfigurableApplicationContext> children = new ArrayList<>();
int n = 0;
for(String feedUrl : feedUrls) {
AnnotationConfigApplicationContext child = new AnnotationConfigApplicationContext();
child.setId("child" + ++n);
children.add(child);
child.setParent(parent);
child.register(DynamicFeedAdapter.class);
StandardEnvironment env = new StandardEnvironment();
Properties props = new Properties();
props.setProperty("feed.url", feedUrl);
PropertiesPropertySource pps = new PropertiesPropertySource("feed", props);
env.getPropertySources().addLast(pps);
child.setEnvironment(env);
child.refresh();
}
System.out.println("Press any key to exit...");
System.in.read();
for (ConfigurableApplicationContext child : children) {
child.close();
}
parent.close();
}
#Bean
public IntegrationFlow aggregateFeeds() {
return IntegrationFlows.from("feedChannel")
.transform(extractLinkFromFeed())
.handle(System.out::println)
.get();
}
#Bean
public MessageChannel feedChannel() {
return new DirectChannel();
}
#Bean
public AbstractPayloadTransformer<SyndEntry, String> extractLinkFromFeed() {
return new AbstractPayloadTransformer<SyndEntry, String>() {
#Override
protected String transformPayload(SyndEntry payload) throws Exception {
return payload.getLink();
}
};
}
}
DynamicFeedAdapter.java
#Configuration
#EnableIntegration
public class DynamicFeedAdapter {
#Value("${feed.url}")
public String feedUrl;
#Bean
public static PropertySourcesPlaceholderConfigurer pspc() {
return new PropertySourcesPlaceholderConfigurer();
}
#Bean
public IntegrationFlow feedAdapter() throws MalformedURLException {
URL url = new URL(feedUrl);
return IntegrationFlows
.from(s -> s.feed(url, "feedTest"),
e -> e.poller(p -> p.fixedDelay(10000)))
.channel("feedChannel")
.get();
}
}
And this works IF and only IF I have one of the urls defined in application.properties as feed.url=[insert url here]. Otherwise it fails telling me 'unable to resolve property {feed.url}'. I suspect what is happening there is that the #Beans defined in DynamicFeedAdapter.java all get singletons eagerly initialized, so aside from the beans being manually created in our for loop in the main method (which work fine because they have feed.url property injected) we have a stray singleton that has been eagerly initialized and if there is no feed.url defined in application.properties then it can't resolve the property and everything goes bang. Now from what I know of Spring, I know it should be possible to #Lazy initialize the beans in DynamicFeedAdapter.java so we don't wind up with this one unwanted stray singleton problem-child. The problem is now...if I just mark the feedAdapter() #Lazy then the beans never get initialized. How do I initialize them myself?
Update - problem solved
Without having tested it, I think the problem is that boot is finding
the DynamicFeedAdapter during its component scan. A simple solution is
to move it to a sibling package. If MonsterFeedApplication is in
com.acme.foo, then put the adapter config class in com.acme.bar. That
way, boot won't consider it "part" of the application
This was indeed the problem. After implementing Gary's suggestion, everything works perfect.
See the answer to this question and its follow up for a similar question about inbound mail adapters.
In essence, each feed adapter is created in a child context that is parameterized.
In that case the child contexts are created in a main() method but there's no reason it couldn't be done in a service invoked by .handle().
I am developing a Neo4j server extension using the Neo4j Framework provided by Graphaware.
I want in my response to send the following object (simplified so that you can see the attributes) :
public class DiffResult {
private Node fileOrFolder;
private Node originalContent;
private Path path;
}
The problem is that the Node object cannot be rendered by Jackson. I have seen a NodeRepresentation class somewhare but I also don't know how to use it properly with my Spring MVC Controller.
I want my nodes to be serialized like in the Neo4j REST Api (cf documentation: http://neo4j.com/docs/stable/rest-api-nodes.html#rest-api-get-node)
I also show you the controller I am using (also simplified).
#Controller
#RequestMapping("/diff")
public class FileSpaceDiffApi {
private final GraphDatabaseService database;
#Autowired
public FileSpaceDiffApi(GraphDatabaseService database) {
this.database = database;
}
#RequestMapping(method = RequestMethod.GET)
#ResponseBody
public List<DiffResult> diff(#QueryParam("fileSpaceId") Long fileSpaceId, #QueryParam("since") Long since) {
List<DiffResult> results = new ArrayList<DiffResult>();
Transaction tx = database.beginTx();
try {
Node startNode = database.getNodeById(fileSpaceId);
DiffResult diffResult = new DiffResult();
diffResult.setFileOrFolder(startNode);
results.add(diffResult);
tx.success();
}
finally {
tx.close();
}
return results;
}
}
Ideally I'd also like to be able to render the Path in JSON.
There is (yet) no capability of easily returning nodes in the same format as Neo4j does. This is mainly because the Neo4j REST API is very generic and thus too chatty and verbose for many use-cases.
I would suggest looking at com.graphaware.api.JsonNode to which you can pass a Neo4j node and some configuration about what will be present in the generated JSON (e.g. whether to include labels, etc.)
You can use it by adding the following to your pom.xml:
<dependency>
<groupId>com.graphaware.neo4j</groupId>
<artifactId>api</artifactId>
<version>${graphaware.version}</version>
</dependency>
As for paths, there is a JsonPath class in neo4j-algorithms, that will help you achieve what you want. We will happily move to the core framework for the next release (that's where it really should be).
Looking around StackOverflow, I see this answer to a similar problem - according to the Twitter4J documentation, TwitterStream#addListener takes a callback function. I have naively written my class as follows:
#Stateless
#LocalBean
public class TwitterListenerThread implements Runnable {
private TwitterStream twitterStream;
public TwitterListenerThread(){}
#EJB private TwitterDispatcher dispatcher;
#Override
public void run() {
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setJSONStoreEnabled(true)
.setOAuthConsumerKey(Properties.getProperty("twitter_OAuthConsumerKey"))
.setOAuthConsumerSecret(Properties.getProperty("twitter_OAuthConsumerSecret"))
.setOAuthAccessToken(Properties.getProperty("twitter_OAuthAccessToken"))
.setOAuthAccessTokenSecret(Properties.getProperty("twitter_OAuthAccessTokenSecret"));
twitterStream = new TwitterStreamFactory(cb.build()).getInstance();
UserStreamListener listener = new UserStreamListener() {
#Override
public void onStatus(Status status) {
dispatcher.dispatch(status);
}
// Standard code
};
twitterStream.addListener(listener);
// Listen for all user activity
String user = Properties.getProperty("twitter-userid");
String[] users = {user};
twitterStream.user(users);
}
}
Now, on my colleague's PC this soon fails with an attempt to invoke when container is undeployed on the dispatcher.dispatch(status); line. I understand the reason as being due to the Twitter4J threading model not playing well with the JavaEE EJB model, but I cannot work out what to do based on the answer presented in the linked answer - how would I use a Message-Driven Bean to listen in to the Twitter stream?
After a little thinking, I worked out that the solution offered was to write a separate application that used just Java SE code to feed, using non-annotated code, a JMS queue with tweets, and then in my main application use a Message-Driven Bean to listen to the queue.
However, I was not satisfied with that work-around, so I searched a little more, and found Issue TFJ-285, Allow for alternative implementations of Dispatcher classes:
Now it is possible to introduce your own dispatcher implementation.
It can be Quartz based, it can be MDB based, and it can be EJB-timer based.
By default, Twitter4J still uses traditional and transient thread based dispatcher.
Implement a class implementing twtitter4j.internal.async.Dispatcher interface
put the class in the classpath
set -Dtwitter4j.async.dispatcherImpl to locate your dispatcher implementation
This is the default implementation on GitHub, so one could replace the:
private final ExecutorService executorService;
with a:
private final ManagedExecutorService executorService;
And, in theory, Bob's your uncle. If I ever get this working, I shall post the code here.
iI'm playing around with the Charts and CDI add-ons for Vaadin at the moment and am trying to inject a mock data source into a Chart class. The data source is a singleton bean that has already had a reference injected into the View that will be displaying the chart but I was under the impression that this shouldn't matter as singletons are application scoped.
The EJB is injected correctly into the view but when the chart class is instantiated, the injection of the data source fails and returns a null reference. I've been using the no-interface facility up until now but even if I do use an interface for the data source, this doesn't make any difference. I'm guessing that there is either a scoping issue or I'm fundamentally misusing/misunderstanding CDI. The other possibility is that I've run into a limitation to the Vaadin CDI add-on as this methodology worked without problems in JSF2.2.
If anyone has any ideas or pointers I'd be really grateful as it's pretty frustrating. Granted this is a quick and dirty implementation but it is a prototype; refactoring to separate concerns (data provision vs building UI components) may well sort the issue but I'd like to understand what's happening here first.
EJB:
#Startup
#Singleton
public class MockDataProvider implements Serializable {
private static final long serialVersionUID = -4789949304830373309L;
private Random rand = new Random();
private Collection<Person> people = new ArrayList<Person>();
private Collection<Address> addresses = new ArrayList<Address>();
private Collection<Evnt> evnts = new ArrayList<Evnt>();
private Collection<TicketType> tickets = new ArrayList<TicketType>();
/**
* Initialize the data for the application
*/
public MockDataProvider() {
}
#PostConstruct
private void init() {
loadAddressData();
loadTicketData();
loadEventData();
loadPersonData();
}
View implementation (injection successful here):
#CDIView(DashboardView.VIEW_ID)
public class DashboardView extends AbstractMVPView implements IDashboardView {
public final static String VIEW_ID = "dashboard";
#Inject
#CDILogger
private Logger logger;
#EJB
MockDataProvider dataProvider;
#Inject
EventsPerMonthChart eventsPerMonthChart;
private Table eventsTable;
private Table peopleTable;
public DashboardView() {
}
Chart class (implemented by DashboardView - EJB injection fails so a null pointer exception is thrown by dataProvider.getEvntCollection.
#Dependent
public class EventsPerMonthChart extends Chart {
#EJB
MockDataProvider dataProvider;
public EventsPerMonthChart() {
super(ChartType.PIE);
setCaption("Events per month");
getConfiguration().setTitle("");
getConfiguration().getChart().setType(ChartType.PIE);
setWidth("100%");
setHeight("90%");
DataSeries series = new DataSeries();
ArrayList<Evnt> events = (ArrayList) dataProvider.getEvntCollection();
OK - it looks like the problem was down to ignorance on my part as I did not understand the contexts where EJB injection is permitted.
The EJB (MockDataProvider) is instantiated by the container and injected into the DashboardView class which, as it was annotated with #CDIView, is also managed by the container. Hence, everything works fine. However, the Chart object was not container managed (despite my misguided addition of #Dependent to try and get the container to "notice" it) - injection into POJOs is not permitted but appears to fail silently which only added to my confusion.
Granted, the code structure is pretty appalling (close coupling, highly dependant and no separation of concerns) and this shoddy approach to prototyping has been responsible for creating the issue. Passing the Charts object the data directly or a via reference to the EJB via a constructor call works without problems.
Good job your learn from your mistakes. At the rate I'm making them, I'm going to be a genius!
The Cross Site Scripting Cheat Sheet has many rules for securing against XSS attacks. I would like to implement those suggestions in my web app which is using Spring MVC + Jackson + JPA + Hibernate Bean Validation. As an example consider the following code that is similar to what I have in my app.
public class MessageJson {
#NotEmpty // Bean Validation annotation
private String title;
#NotEmpty
private String body;
// ... etc getters / setters
}
public class BolgPostController
{
#RequestMapping(value="messages",method=RequestMethod.POST)
public void createMessage(#Valid #RequestBody MessageJson message)
{
// **Question** How do I check that the message title and body don't contain
// nasty javascripts and other junk that should not be there?
// Call services to write data to the datababse
}
#RequestMapping(value="messages",method=RequestMethod.get)
public #ResponseBody List<MessageJson> createMessage()
{
// get data from the database
// **Question** How do I escape all the data in the list of MessageJson before
I send it back to the data.
}
}
I can see the following ways to implement the cheat sheet rules:
Option A Implement them manually in each controller method.
Option B Configure some extension to Spring MVC that can do it for me automatically
Option C Configure Jackson so that it can do it for me since most of my input/output goes through Jackson
I am looking for some example configurations of SpringMVC in any of those three options, with a preference for option B and C.
That would be easiest to do in setters for properties (like setTitle() for title property), when reading JSON.
Or if you are thinking of escaping additional characters (to, say, prevent embedding of HTML markup), have a look at this blog entry: escaping HTML characters in JSON with Jackson.