I'm learning Flyway migration tool, and I don't have clear the concept of checksums. Could someone explain me what is? how is it calculated, or how can it be changed?
I understand the repair command re-calculate the checksum, I do not understand how it differs.
Thanks!
Checksum field in Flyway forming a part of verification mechanism ensuring that migration scripts haven't been changed since they applied to the database. This will guaranty that all instances of your application will have same database structure (content). You can switch off verification, but I will not recommend you to do so. Answering you questions:
What is?
Just google what is checksum. Wikipedia
How is it calculated?
For SQL migrations Flyway uses CRC32 class to calculate the checksum. For exact code see below.
How can it be changed?
The checksum of the migration will be changed once the binary content of you migration modified. If you want to change checksum field in the DB when you need to calculate the checksum for the new version of your migration file and then change the value in the DB. However, I wouldn't recommend to do that. You shouldn't need to do that and the fact that you want to change it may indicate that you doing something wrong. Anyway, the calculation code for the checksum is quite simple (with courtesy of Flyway source code):
/**
* Calculates the checksum of this string.
*
* #param str The string to calculate the checksum for.
* #return The crc-32 checksum of the bytes.
*/
/* private -> for testing */
static int calculateChecksum(Resource resource, String str) {
final CRC32 crc32 = new CRC32();
BufferedReader bufferedReader = new BufferedReader(new StringReader(str));
try {
String line;
while ((line = bufferedReader.readLine()) != null) {
crc32.update(line.getBytes("UTF-8"));
}
} catch (IOException e) {
String message = "Unable to calculate checksum";
if (resource != null) {
message += " for " + resource.getLocation() + " (" + resource.getLocationOnDisk() + ")";
}
throw new FlywayException(message, e);
}
return (int) crc32.getValue();
}
To calculate flyway checksum for arbitrary file, I use the following code:
import java.util.*;
import org.flywaydb.core.internal.resource.filesystem.*;
import org.flywaydb.core.internal.resolver.*;
import java.nio.charset.*;
public class Program {
public static void main( String[] args ) throws Exception{
String filename="/path/to/migration/V8_test.sql";
FileSystemResource r = new FileSystemResource(null, filename,Charset.forName("UTF-8"));
int cs = ChecksumCalculator.calculate(r);
System.out.println(cs);
}
}
Only dependency for this code is org.flywaydb:flyway-core:6.4.1
For version 8.10 #Eugene's answer needs to change a bit:
get_flyway_checksum.java:
import java.util.*;
import org.flywaydb.core.internal.resource.filesystem.*;
import org.flywaydb.core.internal.resolver.*;
import org.flywaydb.core.internal.resource.*;
import org.flywaydb.core.api.Location;
import java.nio.charset.*;
public class Program {
public static void main( String[] args ) throws Exception{
Location loc = null;
String filepath = args[0];
FileSystemResource r = new FileSystemResource(loc, filepath,Charset.forName("UTF-8"), false);
int cs = ChecksumCalculator.calculate(r);
System.out.println(cs);
}
}
To get the checksum:
java -cp {path to your flyway installation}/lib/community/flyway-core-8.1.0.jar get_flyway_checksum.java {filepath}
Related
I am working on a project where I need access method arguments during execution.
Is it possible to print method arguments using byte buddy framework? any sample code on this using javaagent is highly appreciated.
Yes, this is possible. You can use MethodDelegation or Advice to inject your code and then use the #AllArguments annotation to get hold of the actual arguments.
The question is, how do you create your code in your project? You can either use a Java agent with the AgentBuilder or create proxy subclasses using ByteBuddy instances. Refer to the documentation and the mentioned classes javadoc to find out how this is done.
Here is an example of how this can be implemented using MethodDelegation. I use it to measure the execution time of methods. I specifically did not begin to remove the extra code, because I want to more fully reveal the capabilities of Byte Buddy.
package md.leonis.shingler;
import net.bytebuddy.agent.ByteBuddyAgent;
import net.bytebuddy.agent.builder.AgentBuilder;
import net.bytebuddy.implementation.MethodDelegation;
import net.bytebuddy.implementation.bind.annotation.AllArguments;
import net.bytebuddy.implementation.bind.annotation.Origin;
import net.bytebuddy.implementation.bind.annotation.RuntimeType;
import net.bytebuddy.implementation.bind.annotation.SuperCall;
import net.bytebuddy.matcher.ElementMatchers;
import java.lang.instrument.Instrumentation;
import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.concurrent.Callable;
import java.util.stream.Collectors;
public class MeasureMethodTest {
public static void main(String[] args) throws InterruptedException {
premain(ByteBuddyAgent.install());
for (int i = 0; i < 4; i++) {
SampleClass.foo("arg" + i);
}
}
public static void premain(Instrumentation instrumentation) {
new AgentBuilder.Default()
.type(ElementMatchers.nameStartsWith("md.leonis.shingler"))
.transform((builder, type, classLoader, module) ->
builder.method(ElementMatchers.any()).intercept(MethodDelegation.to(AccessInterceptor.class))
).installOn(instrumentation);
}
public static class AccessInterceptor {
#RuntimeType
public static Object intercept(#Origin Method method, #SuperCall Callable<?> callable, #AllArguments Object[] args) throws Exception {
long start = System.nanoTime();
try {
return callable.call();
} finally {
if (method.getAnnotationsByType(Measured.class).length > 0) {
String params = Arrays.stream(args).map(Object::toString).collect(Collectors.joining(", "));
System.out.println(method.getReturnType().getSimpleName() + " " + method.getName() + "("+ params +") took " + ((System.nanoTime() - start) / 1000000) + " ms");
}
}
}
}
public static class SampleClass {
#Measured
static void foo(String s) throws InterruptedException {
Thread.sleep(50);
}
}
}
This example measures the execution time of all methods found in the md.leonis.shingler package and marked with the #Measured annotation.
To run it, you need two libraries: byte-buddy and byte-buddy-agent.
The result of work:
void foo(arg0) took 95 ms
void foo(arg1) took 50 ms
void foo(arg2) took 50 ms
void foo(arg3) took 50 ms
Note that the console displays the values of all arguments passed to the method. This is the answer to the question asked.
Here is the annotation example:
package md.leonis.shingler;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.METHOD)
public #interface Measured {
}
To be honest, I was not able to directly configure filtering by annotations in the Agent. Here is an example (not working):
new AgentBuilder.Default()
.type(ElementMatchers.isAnnotatedWith(Measured.class))
.transform((builder, type, classLoader, module) ->
builder.method(ElementMatchers.any()).intercept(MethodDelegation.to(AccessInterceptor.class))
).installOn(instrumentation);
If someone knows how to do this, please comment below.
can anyone guide me on how to perform a reload of an apache commons configuration2 properties. I'm unable to find any implementation of this anywhere. The apache docs are a bit too abstract. This is what I have so far but it's not working.
CombinedConfiguration cc = new CombinedConfiguration();
Parameters params = new Parameters();
File configFile = new File("config.properties");
File emsFile = new File("anotherconfig.properties");
ReloadingFileBasedConfigurationBuilder<FileBasedConfiguration> configBuilder =
new ReloadingFileBasedConfigurationBuilder<FileBasedConfiguration>(PropertiesConfiguration.class)
.configure(params.fileBased()
.setFile(configFile));
PeriodicReloadingTrigger reloadTrg = new PeriodicReloadingTrigger(configBuilder.getReloadingController(), null, 5, TimeUnit.SECONDS);
reloadTrg.start();
cc.addConfiguration(configBuilder.getConfiguration());
FileBasedConfigurationBuilder<FileBasedConfiguration> emsBuilder =
new FileBasedConfigurationBuilder<FileBasedConfiguration>(PropertiesConfiguration.class)
.configure(params.properties()
.setFile(emsFile));
cc.addConfiguration(emsBuilder.getConfiguration());
DataSource ds = EmsDataSource.getInstance().getDatasource(this);
BasicConfigurationBuilder<DatabaseConfiguration> dbBuilder =
new BasicConfigurationBuilder<DatabaseConfiguration>(DatabaseConfiguration.class);
dbBuilder.configure(
params.database()
.setDataSource(ds)
.setTable("EMS_CONFIG")
.setKeyColumn("KEY")
.setValueColumn("VALUE")
);
cc.addConfiguration(dbBuilder.getConfiguration());
The configuration obtained from a builder is not updated automatically. You need to get the configuration from the builder every time you read it.
From Automatic Reloading of Configuration Sources:
One important point to keep in mind when using this approach to reloading is that reloads are only functional if the builder is used as central component for accessing configuration data. The configuration instance obtained from the builder will not change automagically! So if an application fetches a configuration object from the builder at startup and then uses it throughout its life time, changes on the external configuration file become never visible. The correct approach is to keep a reference to the builder centrally and obtain the configuration from there every time configuration data is needed.
use following code:
#Component
public class ApplicationProperties {
private PropertiesConfiguration configuration;
#PostConstruct
private void init() {
try {
String filePath = PropertiesConstants.PROPERTIES_FILE_PATH;
System.out.println("Loading the properties file: " + filePath);
configuration = new PropertiesConfiguration(filePath);
//Create new FileChangedReloadingStrategy to reload the properties file based on the given time interval
FileChangedReloadingStrategy fileChangedReloadingStrategy = new FileChangedReloadingStrategy();
fileChangedReloadingStrategy.setRefreshDelay(PropertiesConstants.REFRESH_DELAY);
configuration.setReloadingStrategy(fileChangedReloadingStrategy);
} catch (ConfigurationException e) {
e.printStackTrace();
}
}
public String getProperty(String key) {
return (String) configuration.getProperty(key);
}
public void setProperty(String key, Object value) {
configuration.setProperty(key, value);
}
public void save() {
try {
configuration.save();
} catch (ConfigurationException e) {
e.printStackTrace();
}
}
}
I am trying to run the following program.I am using glassfish server 3.1.2 to enable this MDB to run.Then too I am unanble to run it.
package com.mdb;
import javax.jms.ConnectionFactory;
import javax.jms.Queue;
import javax.jms.Connection;
import javax.jms.Session;
import javax.jms.QueueBrowser;
import javax.jms.Message;
import javax.jms.JMSException;
import javax.annotation.Resource;
import java.util.Enumeration;
import javax.ejb.Stateless;
/**
* The MessageBrowser class inspects a queue and displays the messages it
* holds.
*/
#Stateless
public class MessageClient {
#Resource(mappedName = "jms/ConnectionFactory")
private static ConnectionFactory connectionFactory;
#Resource(mappedName = "jms/Queue")
private static Queue queue;
/**
* Main method.
*
* #param args the queue used by the example
*/
public static void main(String[] args) {
Connection connection = null;
try {
System.out.println("1");
connection = connectionFactory.createConnection();
System.out.println("2");
Session session = connection.createSession(
false,
Session.AUTO_ACKNOWLEDGE);
QueueBrowser browser = session.createBrowser(queue);
Enumeration msgs = browser.getEnumeration();
if (!msgs.hasMoreElements()) {
System.out.println("No messages in queue");
} else {
while (msgs.hasMoreElements()) {
Message tempMsg = (Message) msgs.nextElement();
System.out.println("Message: " + tempMsg);
}
}
} catch (JMSException e) {
System.err.println("Exception occurred: " + e.toString());
} finally {
if (connection != null) {
try {
connection.close();
} catch (JMSException e) {
}
}
}
}
}
The problem is I get the follwing exsception upon runing it.
Exception in thread "main" java.lang.NullPointerException
at com.mdb.MessageClient.main(MessageClient.java:35)
What may be the problem here?
What you have build is not a MDB. It's a stateless session bean that browses a queue.
A MDB has the #MessageDriven annotation. It's invoked whenever a message comes in.
Apart from that, you might want to use the "lookup" attribute instead of the "mappedName" one. The latter is from an ancient time when people weren't sure yet about anything, and needed a temporary hack to make things magically work.
Your usage of static fields and the static main method inside a stateless bean make no sense at all. If you're accessing your bean via that main method you're not using the bean at all and you're just calling an isolated global-like method. If anything, this might be the source of your NPE.
The fix isn't really simple. You're seemingly completely confused between Java EE and Java SE, and between instances and static methods.
I have created a servlet to access a database and giving response to a BB application...it was running fine during development...but after loading it on a tomcat server 6.0 after goining live the servlet has to be reloaded every morning on the tomcat server....after that it works fine during the whole day..but the next morning when i request something it gives a blank page as response and my server admin tells the servlet has to be reloaded ...
other application hosted on the server are working fine and do not need a restart...
what might be the problem....
adding the code ..if it helps
package com.ams.servlets;
import java.io.*;
import javax.servlet.*;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.sql.*;
import com.cms.dbaccess.DataAccess;
import com.cms.utils.ApplicationConstants;
import com.cms.utils.ApplicationHelper;
import java.sql.ResultSet;
public class BBRequestProcessorServlet extends HttpServlet {
/**
*
*/String userString;
private static final long serialVersionUID = 1L;
String jsonString = "";
ResultSet rs = null;
Connection connection = null;
Statement statement=null;
public enum db_name
{
//Test
resource_management_db,osms_inventory_db;
}
public void init(ServletConfig config)throws ServletException
{
super.init(config);
System.out.println("Inside init");
}
public void doGet(HttpServletRequest request,HttpServletResponse response)throws ServletException,IOException
{
try{
connection = DataAccess.connectToDatabase("xxx", connection);
statement = DataAccess.createStatement(connection);
statement = connection.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY);
rs = statement.executeQuery("query is here");
}
catch(SQLException e)
{
e.printStackTrace();
}
String db =request.getParameter("db");
System.out.println("DATABASE NAME :"+ db);
if(db.equalsIgnoreCase("xxx")){
//Call to populate JSONArray with the fetch ResultSet data
jsonString = ApplicationHelper.populateJSONArray(rs);
}
response.setContentType(ApplicationConstants.JSON_CONTENT_TYPE);
PrintWriter out = response.getWriter();
out.print(jsonString);
out.flush();
out.close();
System.out.println("json object sent");
try {
rs.close();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
the only errors i could find was
Jul 20, 2012 9:57:24 AM org.apache.catalina.loader.WebappClassLoader validateJarFile
INFO: validateJarFile(/usr/local/tomcat/apache-tomcat-6.0.20/webapps/MobileServlet /WEB-INF/lib/servlet-api.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class
The culprit is most likely the way how you handle external DB resources like the Connection. This problem can happen when you keep a DB Connection open all the time without closing it. When a DB Connection is been opened for a too long time, then the DB will timeout and reclaim it. This is most likely what was happening overnight.
You should redesign your DataAccess and BBRequestProcessorServlet that way so that you are nowhere keeping hold of Connection, Statement and ResultSet as an instance variable, or worse, a static variable of the class. The Connection should be created in the very same scope as where you're executing the SQL query/queries and it should be closed in the finally block of the very same try block as where you've created it.
By the way your jsonString should absolutely also not be declared as an instance variable of the servlet, it's not threadsafe this way.
See also:
Is it safe to use a static java.sql.Connection instance in a multithreaded system?
How do servlets work? Instantiation, sessions, shared variables and multithreading
As to the error which you're seeing in the log, you should definitely remove the offending JAR. See also How do I import the javax.servlet API in my Eclipse project?
I am guessing and will be more clear after seeing your logs.
Its seems like you have putted your servlet-api.jar in the WEB-INF lib but its already in tomcat's lib.
I want to execute my Selenium tests in different languages. Is it possible to change the language of an existing WebDriver at runtime or do I have to recreate the browser instance?
Right now I'm only using Firefox, but I want to execute tests in different browsers at some later point.
In Firefox I set the language like this:
FirefoxProfile profile = new FirefoxProfile();
profile.setPreference("intl.accept_languages", "de");
WebDriver driver = new FirefoxDriver(profile);
I also implemented a WebDriverPool, which holds a WebDriver instance so it can be shared among tests. If the language can only be set at creation time, I could hold an instance for every locale.
All in all I wonder if I miss something here. Why is it so hard to change the language? shouldn't there be a method like WebDriver.setAcceptLanguages(Locale)?
In a nutshell I have these questions:
Why isn't there WebDriver.setAcceptLanguages(Locale)?
How to change the language for the dirrerent WebDrivers?
Can I change the language at runtime?
How did you guys implement your WebDriverPool or do you recreate them everytime?
I ended up creating a WebDriverPool that creates one instance for every combination of WebDriver type (e.g. FirefoxDriver.class) and Locale (e.g. en_US). Maybe this is usful for someone.
public class WebDriverPool {
private Map<String, WebDriver> drivers = new HashMap<String, WebDriver>();
private List<WebDriver> driversInUse = new ArrayList<WebDriver>();
public WebDriverPool() {
Runtime.getRuntime().addShutdownHook(new Thread(){
#Override
public void run(){
for (WebDriver driver : drivers.values())
driver.close();
if (!driversInUse.isEmpty())
throw new IllegalStateException("There are still drivers in use, did someone forget to return it? (size: " + driversInUse.size() + ")");
}
});
}
private WebDriver createFirefoxDriver(Locale locale){
FirefoxProfile profile = new FirefoxProfile();
profile.setPreference("intl.accept_languages", formatLocale(locale));
return new FirefoxDriver(profile);
}
private String formatLocale(Locale locale) {
return locale.getCountry().length() == 0
? locale.getLanguage()
: locale.getLanguage() + "-" + locale.getCountry().toLowerCase();
}
/**
* #param clazz
* #param locale
* #return web driver which can be new or recycled
*/
public synchronized WebDriver getWebDriver(Class<? extends WebDriver> clazz, Locale locale){
String key = clazz.getName() + "-" + locale;
if(!drivers.containsKey(key)){
if(clazz == FirefoxDriver.class){
drivers.put(key, createFirefoxDriver(locale));
}
// TODO create other drivers here ...
// else if(clazz == ChromeDriver.class){
// drivers.put(key, createChromeDriver(locale));
// }
else{
throw new IllegalArgumentException(clazz.getName() + " not supported yet!");
}
}
WebDriver driver = drivers.get(key);
if(driversInUse.contains(driver))
throw new IllegalStateException("This driver is already in use. Did someone forgot to return it?");
driversInUse.add(driver);
return driver;
}
public synchronized void returnWebDriver(WebDriver driver){
driversInUse.remove(driver);
}
}
You can also do it through about:config in firefox. But you need to use Actions to manipulate it.
Below a java piece of code
Actions act = new Actions(webDriver);
webDriver.get("about:config");
// warning screen
act.sendKeys(Keys.RETURN).perform();
// Go directly to the list, don't use the search option, it's not fast enough
act.sendKeys(Keys.TAB).perform();
// Go to the intl.accept_languages option
act.sendKeys("intl.accept_languages").sendKeys(Keys.RETURN).perform();
// fill the alert with your parameters
webDriver.switchTo().alert().sendKeys("fr, fr-fr, en-us, en");
webDriver.switchTo().alert().accept();
I am afraid that the whole idea of WebDriver is to act like browser - so you can change the language of the browser, but you have to change the locale in the Operating system, or hope that the application will do it for you.
For instance - German number format separates decimal number by comma and English one by dot. If you want to test, how the number format behaves in English locale and in German locale, you can do it only by these two approaches:
Change OS locale from German to English or vice versa
Change browser language and hope that application will change the behavior.
To answer your questions:
There is no setLocale on Webdriver, because WebDriver simulates browser, not OS
I would do it like this (Java code):
private WebDriver driver;
public enum Language {en-us, de}
public WebDriver getDriver(Language lang){
String locale = lang.toString();
FirefoxProfile profile = new FirefoxProfile();
profile.setPreference("intl.accept_languages", locale);
driver = new FirefoxDriver(profile);
return driver;
}
#Test
public void TestNumber(){
WebDriver drv = getDriver(Language.en);
drv.get("http://the-site.com");
WebElement el = drv.findElement //... find element
String number = el.getText();
Assert.assertEquals(number, "123.45");
drv.close();
drv = getDriver(Language.de);
drv.get("http://the-site.com");
WebElement el = drv.findElement //... find element
String number = el.getText();
Assert.assertEquals(number, "123,45");
drv.close();
}
I am afraid you have to close the browser and open it again with different language.
I personally create new instance of the browser for each test.
BTW the above bit of code assumes, that the web application changes the way how to show numbers to the user based on browser language.