I would like to know is there a way to get the CPU Usage metrics with Spring Boot Actuator? Im able to see other metrics with /metrics and /health endpoints but not getting the CPU Usage.
I want to avoid writing an extra class just to see the CPU Usage.
Any idea?
Thanks
Just checked and I found this actuator... /actuator/metrics/process.cpu.usage
It outputs the following:
{
name: "process.cpu.usage",
description: "The "recent cpu usage" for the Java Virtual Machine process",
baseUnit: null,
measurements: [
{
statistic: "VALUE",
value: 0.0001742149747252696
}
],
availableTags: [ ]
}
Currently using Spring Boot version 2.2.2.RELEASE.
Spring Boot 2 actuator solution (building on #diginoise's code to measure CPU load), registering a Gauge with a function to measure the value when requested (no need to start Threads or schedule timers):
#Component
public class CpuMetrics {
private final static String METRICS_NAME = "process.cpu.load";
#Autowired
private MeterRegistry meterRegistry;
#PostConstruct
public void init() {
Gauge.builder(METRICS_NAME, this, CpuMetrics::getProcessCpuLoad)
.baseUnit("%")
.description("CPU Load")
.register(meterRegistry);
}
public Double getProcessCpuLoad() {
try {
MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
ObjectName name = ObjectName.getInstance("java.lang:type=OperatingSystem");
AttributeList list = mbs.getAttributes(name, new String[]{"ProcessCpuLoad"});
return Optional.ofNullable(list)
.map(l -> l.isEmpty() ? null : l)
.map(List::iterator)
.map(Iterator::next)
.map(Attribute.class::cast)
.map(Attribute::getValue)
.map(Double.class::cast)
.orElse(null);
} catch (Exception ex) {
return null;
}
}
}
The CPU metrics will then be available at /actuator/metrics/process.cpu.load:
{
"name": "process.cpu.load",
"description": "CPU Load",
"baseUnit": "%",
"measurements": [
{
"statistic": "VALUE",
"value": 0.09767676212004521
}
],
"availableTags": []
}
Unfortunately there isn't a CPU metric available via Spring Boot Actuator.
Fortunately you could write your own.
Just create a measuring bean which fulfills the following:
It has access to GaugeService as it will be tracking one value.
#Autowired
private GaugeService gaugeService;
Creates a thread which calls routine to measure process' CPU load:
#PostConstruct
public void startMeasuring() {
new Thread() {
#Override
public void run() {
gaugeService.submit("process.cpu.load", getProcessCpuLoad());
Thread.sleep(2000); //measure every 2sec.
}
}.start();
}
Has a routine which gets CPU load for your process using MxBeans:
public static double getProcessCpuLoad() throws Exception {
MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
ObjectName name = ObjectName.getInstance("java.lang:type=OperatingSystem");
AttributeList list = mbs.getAttributes(name, new String[]{ "ProcessCpuLoad" });
if (list.isEmpty()) return Double.NaN;
Attribute att = (Attribute)list.get(0);
Double value = (Double)att.getValue();
// usually takes a couple of seconds before we get real values
if (value == -1.0) return Double.NaN;
// returns a percentage value with 1 decimal point precision
return ((int)(value * 1000) / 10.0);
}
You could also extract the system wide CPU load using this method.
Hope this helps.
Related
Is there a way to specify the dlt used when retrying with spring-kafka #RetryableTopic.
I use a listener with the following configuration :
#RetryableTopic(
attempts = "4",
backoff = #Backoff(delay = 1000),
autoCreateTopics = "false",
topicSuffixingStrategy = TopicSuffixingStrategy.SUFFIX_WITH_DELAY_VALUE,
fixedDelayTopicStrategy = FixedDelayStrategy.SINGLE_TOPIC)
#KafkaListener(topics = "${spring.kafka.template.default-topic}")
This retries using a single topic but uses my main topic +"_dlt" for exhausted retries even though I have a dead letter topic with a different name configured at :
spring:
kafka:
consumer:
template:
dead-letter-topic: its_dead_jim
I've used a DeadLetterPublishingRecoverer in the past and have implemented the dlt resolver function but I don't see a way to override the default behavior in the documentation for RetryableTopic. I've looked at RetryTopicConfigurationBuilder and RetryTopicConfigurer
but nothing seem applicable to change the DLT name.
I am not sure why you think there is such a property on the template.
See the documentation.
Extend RetryTopicConfigurationSupport in a #Configuration class and...
Custom naming strategies
More complex naming strategies can be accomplished by registering a bean that implements RetryTopicNamesProviderFactory. The default implementation is SuffixingRetryTopicNamesProviderFactory and a different implementation can be registered in the following way:
#Override
protected RetryTopicComponentFactory createComponentFactory() {
return new RetryTopicComponentFactory() {
#Override
public RetryTopicNamesProviderFactory retryTopicNamesProviderFactory() {
return new CustomRetryTopicNamesProviderFactory();
}
};
}
As an example the following implementation, in addition to the standard suffix, adds a prefix to retry/dl topics names:
public class CustomRetryTopicNamesProviderFactory implements RetryTopicNamesProviderFactory {
#Override
public RetryTopicNamesProvider createRetryTopicNamesProvider(
DestinationTopic.Properties properties) {
if(properties.isMainEndpoint()) {
return new SuffixingRetryTopicNamesProvider(properties);
}
else {
return new SuffixingRetryTopicNamesProvider(properties) {
#Override
public String getTopicName(String topic) {
return "my-prefix-" + super.getTopicName(topic);
}
};
}
}
}
I could not find a definitive answer to whether it is safe to spawn threads within session-scoped JSF managed beans. The thread needs to call methods on the stateless EJB instance (that was dependency-injected to the managed bean).
The background is that we have a report that takes a long time to generate. This caused the HTTP request to time-out due to server settings we can't change. So the idea is to start a new thread and let it generate the report and to temporarily store it. In the meantime the JSF page shows a progress bar, polls the managed bean till the generation is complete and then makes a second request to download the stored report. This seems to work, but I would like to be sure what I'm doing is not a hack.
Check out EJB 3.1 #Asynchronous methods. This is exactly what they are for.
Small example that uses OpenEJB 4.0.0-SNAPSHOTs. Here we have a #Singleton bean with one method marked #Asynchronous. Every time that method is invoked by anyone, in this case your JSF managed bean, it will immediately return regardless of how long the method actually takes.
#Singleton
public class JobProcessor {
#Asynchronous
#Lock(READ)
#AccessTimeout(-1)
public Future<String> addJob(String jobName) {
// Pretend this job takes a while
doSomeHeavyLifting();
// Return our result
return new AsyncResult<String>(jobName);
}
private void doSomeHeavyLifting() {
try {
Thread.sleep(SECONDS.toMillis(10));
} catch (InterruptedException e) {
Thread.interrupted();
throw new IllegalStateException(e);
}
}
}
Here's a little testcase that invokes that #Asynchronous method several times in a row.
Each invocation returns a Future object that essentially starts out empty and will later have its value filled in by the container when the related method call actually completes.
import javax.ejb.embeddable.EJBContainer;
import javax.naming.Context;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
public class JobProcessorTest extends TestCase {
public void test() throws Exception {
final Context context = EJBContainer.createEJBContainer().getContext();
final JobProcessor processor = (JobProcessor) context.lookup("java:global/async-methods/JobProcessor");
final long start = System.nanoTime();
// Queue up a bunch of work
final Future<String> red = processor.addJob("red");
final Future<String> orange = processor.addJob("orange");
final Future<String> yellow = processor.addJob("yellow");
final Future<String> green = processor.addJob("green");
final Future<String> blue = processor.addJob("blue");
final Future<String> violet = processor.addJob("violet");
// Wait for the result -- 1 minute worth of work
assertEquals("blue", blue.get());
assertEquals("orange", orange.get());
assertEquals("green", green.get());
assertEquals("red", red.get());
assertEquals("yellow", yellow.get());
assertEquals("violet", violet.get());
// How long did it take?
final long total = TimeUnit.NANOSECONDS.toSeconds(System.nanoTime() - start);
// Execution should be around 9 - 21 seconds
assertTrue("" + total, total > 9);
assertTrue("" + total, total < 21);
}
}
Example source code
Under the covers what makes this work is:
The JobProcessor the caller sees is not actually an instance of JobProcessor. Rather it's a subclass or proxy that has all the methods overridden. Methods that are supposed to be asynchronous are handled differently.
Calls to an asynchronous method simply result in a Runnable being created that wraps the method and parameters you gave. This runnable is given to an Executor which is simply a work queue attached to a thread pool.
After adding the work to the queue, the proxied version of the method returns an implementation of Future that is linked to the Runnable which is now waiting on the queue.
When the Runnable finally executes the method on the real JobProcessor instance, it will take the return value and set it into the Future making it available to the caller.
Important to note that the AsyncResult object the JobProcessor returns is not the same Future object the caller is holding. It would have been neat if the real JobProcessor could just return String and the caller's version of JobProcessor could return Future<String>, but we didn't see any way to do that without adding more complexity. So the AsyncResult is a simple wrapper object. The container will pull the String out, throw the AsyncResult away, then put the String in the real Future that the caller is holding.
To get progress along the way, simply pass a thread-safe object like AtomicInteger to the #Asynchronous method and have the bean code periodically update it with the percent complete.
Introduction
Spawning threads from within a session scoped managed bean is not necessarily a hack as long as it does the job you want. But spawning threads at its own needs to be done with extreme care. The code should not be written that way that a single user can for example spawn an unlimited amount of threads per session and/or that the threads continue running even after the session get destroyed. It would blow up your application sooner or later.
The code needs to be written that way that you can ensure that an user can for example never spawn more than one background thread per session and that the thread is guaranteed to get interrupted whenever the session get destroyed. For multiple tasks within a session you need to queue the tasks.
Also, all those threads should preferably be served by a common thread pool so that you can put a limit on the total amount of spawned threads at application level.
Managing threads is thus a very delicate task. That's why you'd better use the built-in facilities rather than homegrowing your own with new Thread() and friends. The average Java EE application server offers a container managed thread pool which you can utilize via among others EJB's #Asynchronous and #Schedule. To be container independent (read: Tomcat-friendly), you can also use the Java 1.5's Util Concurrent ExecutorService and ScheduledExecutorService for this.
Below examples assume Java EE 6+ with EJB.
Fire and forget a task on form submit
#Named
#RequestScoped // Or #ViewScoped
public class Bean {
#EJB
private SomeService someService;
public void submit() {
someService.asyncTask();
// ... (this code will immediately continue without waiting)
}
}
#Stateless
public class SomeService {
#Asynchronous
public void asyncTask() {
// ...
}
}
Asynchronously fetch the model on page load
#Named
#RequestScoped // Or #ViewScoped
public class Bean {
private Future<List<Entity>> asyncEntities;
#EJB
private EntityService entityService;
#PostConstruct
public void init() {
asyncEntities = entityService.asyncList();
// ... (this code will immediately continue without waiting)
}
public List<Entity> getEntities() {
try {
return asyncEntities.get();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new FacesException(e);
} catch (ExecutionException e) {
throw new FacesException(e);
}
}
}
#Stateless
public class EntityService {
#PersistenceContext
private EntityManager entityManager;
#Asynchronous
public Future<List<Entity>> asyncList() {
List<Entity> entities = entityManager
.createQuery("SELECT e FROM Entity e", Entity.class)
.getResultList();
return new AsyncResult<>(entities);
}
}
In case you're using JSF utility library OmniFaces, this could be done even faster if you annotate the managed bean with #Eager.
Schedule background jobs on application start
#Singleton
public class BackgroundJobManager {
#Schedule(hour="0", minute="0", second="0", persistent=false)
public void someDailyJob() {
// ... (runs every start of day)
}
#Schedule(hour="*/1", minute="0", second="0", persistent=false)
public void someHourlyJob() {
// ... (runs every hour of day)
}
#Schedule(hour="*", minute="*/15", second="0", persistent=false)
public void someQuarterlyJob() {
// ... (runs every 15th minute of hour)
}
#Schedule(hour="*", minute="*", second="*/30", persistent=false)
public void someHalfminutelyJob() {
// ... (runs every 30th second of minute)
}
}
Continuously update application wide model in background
#Named
#RequestScoped // Or #ViewScoped
public class Bean {
#EJB
private SomeTop100Manager someTop100Manager;
public List<Some> getSomeTop100() {
return someTop100Manager.list();
}
}
#Singleton
#ConcurrencyManagement(BEAN)
public class SomeTop100Manager {
#PersistenceContext
private EntityManager entityManager;
private List<Some> top100;
#PostConstruct
#Schedule(hour="*", minute="*/1", second="0", persistent=false)
public void load() {
top100 = entityManager
.createNamedQuery("Some.top100", Some.class)
.getResultList();
}
public List<Some> list() {
return top100;
}
}
See also:
Spawning threads in a JSF managed bean for scheduled tasks using a timer
I tried this and works great from my JSF managed bean
ExecutorService executor = Executors.newFixedThreadPool(1);
#EJB
private IMaterialSvc materialSvc;
private void updateMaterial(Material material, String status, Location position) {
executor.execute(new Runnable() {
public void run() {
synchronized (position) {
// TODO update material in audit? do we need materials in audit?
int index = position.getMaterials().indexOf(material);
Material m = materialSvc.getById(material.getId());
m.setStatus(status);
m = materialSvc.update(m);
if (index != -1) {
position.getMaterials().set(index, m);
}
}
}
});
}
#PreDestroy
public void destory() {
executor.shutdown();
}
I have a simple application with a single JavaFX window. I'm sending in data to an Azure IoTHub inside a for loop. This for loop is in a JavaFX Task, and the for loop has a small delay (Thread.sleep(300)) so progress can be shown on the UI. I have 2 labels I want to update during the data transmission, always showing the latest sent in data. I have the following helper class for this:
public class DataHelper {
private StringProperty date = new SimpleStringProperty();
private StringProperty count = new SimpleStringProperty();
public DataHelper() {
}
public DataHelper(String date, String count) {
this.date.setValue(date);
this.count.setValue(count);
}
//getters and setters
}
And here is my sendErrorsToHub method inside my UI controller class:
private void sendErrorsToHub(List<TruckErrorForCloud> toCloud) {
DataHelper dataHelper = new DataHelper("", "");
Task task = new Task<DataHelper>() {
#Override
public DataHelper call() {
try {
int i = 0;
for (TruckErrorForCloud error : toCloud) {
Thread.sleep(300);
i++;
String strMessage = Utility.toPrettyJson(null, error);
if (strMessage != null) {
Message msg = new Message(strMessage);
msg.setMessageId(java.util.UUID.randomUUID().toString());
client.sendEventAsync(msg, null, null);
}
updateProgress(i, toCloud.size());
DataHelper dh = new DataHelper(error.getErrorTimeStamp().substring(0, error.getErrorTimeStamp().length() - 9),
String.valueOf(error.getCount()));
updateValue(dh);
}
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
#Override
protected void updateValue(DataHelper value) {
super.updateValue(value);
dataHelper.setDate(value.getDate());
dataHelper.setCount(value.getCount());
}
//succeeded method omitted
};
dateValue.textProperty().bind(dataHelper.dateProperty());
countValue.textProperty().bind(dataHelper.countProperty());
progressBar.progressProperty().bind(task.progressProperty());
new Thread(task).start();
}
When I run the application, I constantly get IllegalStateException: Not on FX application threadexceptions, inside the updateValue method. As far as I understand the documentation, the whole point of the updateValue method, that it runs on the Application thread, and it can be used to pass a custom object, which can be used to update the UI.
What am I doing wrong then?
The bottom of the stacktrace with my classes is the following:
at eu.mantis.still_rca_simulator.gui.DataHelper.setDate(DataHelper.java:28)
at eu.mantis.still_rca_simulator.gui.GuiController$1.updateValue(GuiController.java:166)
at eu.mantis.still_rca_simulator.gui.GuiController$1.call(GuiController.java:155)
at eu.mantis.still_rca_simulator.gui.GuiController$1.call(GuiController.java:138)
(138 is the line Task task = new Task(), 155 updateValue(dh);, 166 dataHelper.setDate(value.getDate());)
updateValue does not automatically run on the application thread and it's not necessary to run it on the application thread since it takes care of updating the value property of Task on the application thread.
Your code in the overridden version updateValue executes logic on the background thread that needs to be run on the application thread though:
dataHelper.setDate(value.getDate());
dataHelper.setCount(value.getCount());
The bindings result in the text properties being updated from the background thread since the above code runs on the background thread.
In this case I recommend using a immutable DataHelper class and updating the ui using a listener to the value property:
Remove the updateValue override and the dataHelper local variable, initialize the gui with empty strings, if necessary, declare task as Task<DataHelper> task and do the following to update the gui:
task.valueProperty().addListener((o, oldValue, newValue) -> {
if (newValue != null) {
dateValue.setText(newValue.getDate());
countValue.setText(newValue.getCount());
}
});
You may also use Platform.runLater for those updates, since they don't happen frequently enough to result in issues that could be the result of using Platform.runLater too frequently.
I have already followed the guide here. I have tried both the config and "in code" approach of initializing and registering our telemetry processor. My goal is to filter out some HTTP responses so that those don't make their way to the sampled data. I haven't had any success. While our processor is initialized on app start, the Process method is never hit. Also, I already made sure that there is an InstrumentationKey in the config and that I'm using the correct key. What else am I missing?
This is what I have:
public class MyTelemetryProcessor : ITelemetryProcessor
{
private ITelemetryProcessor Next { get; set; }
// You can pass values from .config
public string MyParamFromConfigFile { get; set; }
// Link processors to each other in a chain.
public MyTelemetryProcessor(ITelemetryProcessor next)
{
this.Next = next; <-- this is always hit indicating this processor is active
}
public void Process(ITelemetry item)
{
// To filter out an item, just return
if (!OKtoSend(item)) { return; } <-- breakpoint here is never hit
// Modify the item if required
ModifyItem(item);
this.Next.Process(item);
}
private bool OKtoSend(ITelemetry item) <-- and consequently this method is never hit
{
var request = item as RequestTelemetry; <-- breakpoint here is never hit
// some more code goes here
return request.Success.GetValueOrDefault(false);
}
// Example: replace with your own modifiers.
private void ModifyItem(ITelemetry item)
{
item.Context.Properties.Add("app-version", "1." + MyParamFromConfigFile);
}
}
And this is how it is registered. I can see this being hit during debugging when the app starts up:
var builder = TelemetryConfiguration.Active.TelemetryProcessorChainBuilder;
builder.Use((next) => new MyTelemetryProcessor (next));
builder.Build();
In aspnetcore, my solution was to use :
services.AddApplicationInsightsTelemetryProcessor(typeof(BasicTelemetryFilter));
(using the regular CreateWebHostBuilder :
WebHost.CreateDefaultBuilder(args)
.UseApplicationInsights()
.UseStartup<Startup>();
)
In my project based on Spring Boot 1.3.3, I integrated MyBatis with mybatis-spring-boot-starter 1.1.1 as persistence layer, all CRUD operation seems working fine separately, but the integration tests failed and I found the DB operation gets blocked in asynchronous task.
The test code looks like this:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration(classes = SapiApplication.class)
#Transactional
public class OrderIntegrationTest {
#Test
public void shouldUpdateOrder() throws InterruptedException{
Order order1 = getOrder1();
orderService.createOrder(order1);
Order order1updated = getOrder1Updated();
orderService.updateOrderAsync(order1updated);
Thread.sleep(1000l);
log.info("find the order!");
Order order1Db = orderService.findOrderById(order1.getOrderId());
log.info("found the order!");
assertEquals("closed", order1Db.getStatus());
}
}
The expected execution order is createOrder() -> updateOrderAsync() -> findOrderById(), but actually the execution order is createOrder() -> updateOrderAsync() started and blocked -> findOrderById() -> updateOrderAsync() continued and ended.
Log:
16:23:04.261 [executor1-1] INFO c.s.api.web.service.OrderServiceImpl - updating order: 2884384
16:23:05.255 [main] INFO c.s.a.w.service.OrderIntegrationTest - find the order!
16:23:05.280 [main] INFO c.s.a.w.service.OrderIntegrationTest - found the order!
16:23:05.299 [executor1-1] INFO c.s.api.web.service.OrderServiceImpl - updated order: 2884384
Other related code:
#Service
public class OrderServiceImpl implements OrderService {
#Autowired
private OrderDao orderDao;
#Async("executor1")
#Override
public void updateOrderAsync(Order order){
log.info("updating order: {}", order.getOrderId());
orderDao.updateOrder(order);
log.info("updated order: {}", order.getOrderId());
}
}
The DAO:
public interface OrderDao {
public int updateOrder(Order order);
public int createOrder(Order order);
public Order findOrderById(String orderId);
}
The Gradle dependencies:
dependencies {
compile 'org.springframework.boot:spring-boot-starter-jdbc'
compile 'org.springframework.boot:spring-boot-starter-security'
compile 'org.springframework.boot:spring-boot-starter-web'
compile 'org.springframework.boot:spring-boot-starter-actuator'
compile 'org.mybatis.spring.boot:mybatis-spring-boot-starter:1.1.1'
compile 'ch.qos.logback:logback-classic:1.1.2'
compile 'org.springframework.boot:spring-boot-configuration-processor'
runtime 'mysql:mysql-connector-java'
providedRuntime 'org.springframework.boot:spring-boot-starter-tomcat'
testCompile 'org.springframework.boot:spring-boot-starter-test'
testCompile "org.springframework.security:spring-security-test"
}
The Spring configuration:
#SpringBootApplication
#EnableAsync
#EnableCaching
#EnableScheduling
#MapperScan("com.sapi.web.dao")
public class SapiApplication {
#Bean(name = "executor1")
protected Executor taskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(5);
executor.setMaxPoolSize(100);
return executor;
}
#Bean
#Primary
#ConfigurationProperties(prefix = "datasource.primary")
public DataSource numberMasterDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "secondary")
#ConfigurationProperties(prefix = "datasource.secondary")
public DataSource provisioningDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "jdbcTpl")
public JdbcTemplate jdbcTemplate(#Qualifier("secondary") DataSource dsItems) {
return new JdbcTemplate(dsItems);
}
public static void main(String[] args) {
SpringApplication.run(SapiApplication.class, args);
}
}
The properties:
mybatis.mapper-locations=classpath*:com/sapi/web/dao/*Mapper.xml
mybatis.type-aliases-package=com.sapi.web.vo
datasource.primary.driver-class-name=com.mysql.jdbc.Driver
datasource.primary.url=jdbc:mysql://10.0.6.202:3306/sapi
datasource.primary.username=xxx
datasource.primary.password=xxx
datasource.primary.maximum-pool-size=80
datasource.primary.max-idle=10
datasource.primary.max-active=150
datasource.primary.max-wait=10000
datasource.primary.min-idle=5
datasource.primary.initial-size=5
datasource.primary.validation-query=SELECT 1
datasource.primary.test-on-borrow=false
datasource.primary.test-while-idle=true
datasource.primary.time-between-eviction-runs-millis=18800
datasource.primary.jdbc-interceptors=ConnectionState;SlowQueryReport(threshold=100)
datasource.secondary.url = jdbc:mysql://10.0.6.202:3306/xdb
datasource.secondary.username = xxx
datasource.secondary.password = xxx
datasource.secondary.driver-class-name = com.mysql.jdbc.Driver
logging.level.org.springframework.web=DEBUG
The problem you see is caused by the fact that the whole test method shouldUpdateOrder is executed in one transaction. This means that any update operation that is executed in the thread that runs shouldUpdateOrder locks the record for the whole duration of the transaction (that is till exit from test method) and that record cannot be updated by another concurrent transaction (that is executed in async method).
To solve the issue you need to change transactions boundaries. In your case the correct way to emulate real life usage is to
create order in one transaction and finish the transaction
update order in another transaction
check that update is executed as expected in yet another transaction