I'm writing a wrapper around hiredis to enable publish / subscribe functionality with reconnects should a redis node go down.
I'm using the asynchronous redis API.
So I have a test harness that sets up a publisher and subscriber. The harness then shuts down the slave VM from which the subscriber is reading.
However, the disconnect callback isn't called until much later (when I'm destructing the Subscription object that contains the corresponding redisAsyncContext.
I thought that the solution to this might be using tcp keepalive.
So I found that there's a suitable redis function in net.h:
int redisKeepAlive (redisContext* c, int interval);
However, the following appears to show that the redisKeepAlive function has been omitted from the library on purpose:
$ nm libhiredis.a --demangle | grep redisKeepAlive
0000000000000030 T redisKeepAlive
U redisKeepAlive
$ nm libhiredis.a -u --demangle | grep redisKeepAlive
U redisKeepAlive
Certainly when I try to use the call, the linker complains:
Subscription.cpp:167: undefined reference to `redisKeepAlive(redisContext*, int)'
collect2: error: ld returned 1 exit status
Am I out of luck - is there a way to set the TCP keepalive interval on the Hiredis async context?
Update
I've found this:
int redisEnableKeepAlive(redisContext *c);
But setting this on the asyncContext->c and adjusting REDIS_KEEPALIVE_INTERVAL seems to have no effect.
I found that the implementation of redisKeepAlive contains code that shows how to get direct access to the underlying socket descriptor:
int redisKeepAlive(redisContext *c, int interval) {
int val = 1;
int fd = c->fd;
if (setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, &val, sizeof(val)) == -1){
__redisSetError(c,REDIS_ERR_OTHER,strerror(errno));
return REDIS_ERR;
}
Maybe this'll help someone..
Related
CLIPS version: 6.31 .
language: c++ clips C API .
I get a coredump file when execute ProfileInfoCommand after EnvRun.
Why is this error happening? Is there any usage error here? Or this is a bug?
The c++ code 1:
#define PROFILING_FUNCTIONS 1
// ...
EnvReset(clips);
// ...
EnvLoadFactsFromString(clips, facts.str().c_str(), -1);
// ...
EnvRun(clips, 100000);
ProfileInfoCommand(clips);
I know if PROFILING_FUNCTIONS is defined as 1, the EnvRun function will start profile automatically.So I use ProfileInfoCommand after EnvRun,but the coredump has occurred!
And I also tried using another method,but the process also generated a core dump(the same backtrace like the c++ code 1).
The c++ code 2:
EnvReset(clips);
Profile(clips, "constructs");
// ...
EnvLoadFactsFromString(clips, facts.str().c_str(), -1);
// ...
EnvRun(clips, max_iters);
Profile(clips, "off");
ProfileInfoCommand(clips);
The coredump file is following:
Core was generated by `/mnt/home/worker/project/ae-arbiter/dist/bin/arbiter-8003 --flagfile=flags.'.
Program terminated with signal 11, Segmentation fault.
#0 0x0000000000bc6b80 in EnvRtnArgCount (theEnv=Cannot access memory at address 0x7f879c3f6af8
) at /mnt/home/worker/project/ae-arbiter/src/clips/argacces.cc:306
306 for (argPtr = EvaluationData(theEnv)->CurrentExpression->argList;
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.212.el6.x86_64
(gdb) bt
#0 0x0000000000bc6b80 in EnvRtnArgCount (theEnv=0x7f85e6454f70) at /mnt/home/worker/project/ae-arbiter/src/clips/argacces.cc:306
#1 0x0000000000bc6bcd in EnvArgCountCheck (theEnv=0x7f85e6454f70, functionName=0xda1188 "profile", countRelation=2, expectedNumber=1) at /mnt/home/worker/project/ae-arbiter/src/clips/argacces.cc:337
#2 0x0000000000c40803 in ProfileInfoCommand (theEnv=0x7f85e6454f70) at /mnt/home/worker/project/ae-arbiter/src/clips/proflfun.cc:245
#3 0x0000000000b62d12 in arbiter::lib::ClipsModuleExecute (clips=0x7f85e6454f70, features=..., max_iters=100000, result_func=..., module_name=..., halt=#0x7f879c3f6fdc)
at /mnt/home/worker/project/ae-arbiter/src/lib/clips-utils.cc:357
...
...
Setting PROFILING_FUNCTIONS to 1 does not automatically profile code when EnvRun is called. It determines whether the profiling functions are included in the CLIPS executable. See Section 2.2 of the Advanced Programming Guide. The functions available for profiling from the CLIPS command are documented in Section 13.15, Profiling Commands, of the Basic Programming Guide. The ProfileInfoCommand can't be called directly. If you want to call a CLIPS command/function that isn't part of the API described in the Advanced Programming Guide, use the EnvEval API:
DATA_OBJECT returnValue;
EnvEval(theEnv,"(profile-info)",&returnValue);
I get the below error when i try to open and download .realm file in /tmp directory of serverless framework.
{"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"Error: posix_fallocate() failed: Operation not permitted" }
Below is the code:
let realm = new Realm({path: '/tmp/custom.realm', schema: [schema1, schema2]});
realm.write(() => {
console.log('completed==');
});
EDIT: this might soon be finally fixed in Realm-Core: see issue 4957.
In case you'll run into this problem elsewhere, here's a workaround.
This caused by AWS Lambda not supporting the fallocate and fallocate64 system calls. Instead of returning the correct error code in this case, which would be EINVAL for not supported on this file system, Amazon has blocked the system call so that it returns EPERM. Realm-Core has code that handles EINVAL return value correctly but will be bewildered by the unexpected EPERM returned from the system call.
The solution is to add a small shared library as a layer to the lambda: compile the following C file on Linux machine or inside lambda-ci Docker image:
#include <errno.h>
#include <fcntl.h>
int posix_fallocate(int __fd, off_t __offset, off_t __len) {
return EINVAL;
}
int posix_fallocate64(int __fd, off_t __offset, off_t __len) {
return EINVAL;
}
Now, compile this to a shared object with something like
gcc -shared fix.c -o fix.so
Then add it to a root of a ZIP file:
zip layer.zip fix.so
Create a new lambda layer from this zip
Add the lambda layer to your lambda function
Finally make the shared object be loaded by configuring the environment value LD_PRELOAD with value /opt/fix.so to your Lambda.
Enjoy.
I need ,before spawning a number of workers processes as shown below, to check if this number is available so that the below code will not crash if the requested slots is not available.
int numworkers = settings.Parallelism + 1; //omp_get_num_procs();
MPI_Comm_spawn("./processes/montecarlo", MPI_ARGV_NULL, numworkers,
MPI_INFO_NULL,
0, MPI_COMM_SELF, &workercomm, MPI_ERRCODES_IGNORE);
How to check available slots for mpi?
This is happening in the context of a service accepting several requests:
let us suppose:Total available slots: 13
REQ1: spawn 5 processes
Req2: spawn another 5 proc
Req3: will try to spawn 5 proc , but will crash because only 3 is available. how to check that only 3 is available?
or otherwise how to handle the crash that results from the non availability of resources. this crash is killing the service.
you can simply ask MPI_Comm_spawn() to return with an error code instead of aborting the application.
MPI_Comm_set_errhandler(MPI_COMM_SELF, MPI_ERRORS_RETURN);
int res = MPI_Comm_spawn("./processes/montecarlo", MPI_ARGV_NULL, numworkers,
MPI_INFO_NULL, 0, MPI_COMM_SELF, &workercomm, MPI_ERRCODES_IGNORE);
if (MPI_SUCCESS != res) {
// MPI_Comm_spawn failed
}
I'm looking for a way to mimick a terminal for some automated testing: i.e. start a process and then interact with it via sending data to stdin and reading from stdout. E.g. sending some lines of input to stdin including ctrl-c and ctrl-\ which should result in sending signals to the process.
Using std::process::Commannd I'm able to send input to e.g. cat and I'm also seeing its output on stdout, but sending ctrl-c (as I understand that is 3) does not cause SIGINT sent to the shell. E.g. this program should terminate:
use std::process::{Command, Stdio};
use std::io::Write;
fn main() {
let mut child = Command::new("sh")
.arg("-c").arg("-i").arg("cat")
.stdin(Stdio::piped())
.spawn().unwrap();
let mut stdin = child.stdin.take().unwrap();
stdin.write(&[3]).expect("cannot send ctrl-c");
child.wait();
}
I suspect the issue is that sending ctrl-c needs the some tty and via sh -i it's only in "interactive mode".
Do I need to go full fledged and use e.g. termion or ncurses?
Update: I confused shell and terminal in the original question. I cleared this up now. Also I mentioned ssh which should have been sh.
The simplest way is to directly send the SIGINT signal to the child process. This can be done easily using nix's signal::kill function:
// add `nix = "0.15.0"` to your Cargo.toml
use std::process::{Command, Stdio};
use std::io::Write;
fn main() {
// spawn child process
let mut child = Command::new("cat")
.stdin(Stdio::piped())
.spawn().unwrap();
// send "echo\n" to child's stdin
let mut stdin = child.stdin.take().unwrap();
writeln!(stdin, "echo");
// sleep a bit so that child can process the input
std::thread::sleep(std::time::Duration::from_millis(500));
// send SIGINT to the child
nix::sys::signal::kill(
nix::unistd::Pid::from_raw(child.id() as i32),
nix::sys::signal::Signal::SIGINT
).expect("cannot send ctrl-c");
// wait for child to terminate
child.wait().unwrap();
}
You should be able to send all kinds of signals using this method. For more advanced "interactivity" (e.g. child programs like vi that query terminal size) you'd need to create a pseudoterminal like #hansaplast did in his solution.
After a lot of research I figured out it's not too much work to do the pty fork myself. There's pty-rs, but it has bugs and seems unmaintained.
The following code needs pty module of nix which is not yet on crates.io, so Cargo.toml needs this for now:
[dependencies]
nix = {git = "https://github.com/nix-rust/nix.git"}
The following code runs cat in a tty and then writes/reads from it and sends Ctrl-C (3):
extern crate nix;
use std::path::Path;
use nix::pty::{posix_openpt, grantpt, unlockpt, ptsname};
use nix::fcntl::{O_RDWR, open};
use nix::sys::stat;
use nix::unistd::{fork, ForkResult, setsid, dup2};
use nix::libc::{STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO};
use std::os::unix::io::{AsRawFd, FromRawFd};
use std::io::prelude::*;
use std::io::{BufReader, LineWriter};
fn run() -> std::io::Result<()> {
// Open a new PTY master
let master_fd = posix_openpt(O_RDWR)?;
// Allow a slave to be generated for it
grantpt(&master_fd)?;
unlockpt(&master_fd)?;
// Get the name of the slave
let slave_name = ptsname(&master_fd)?;
match fork() {
Ok(ForkResult::Child) => {
setsid()?; // create new session with child as session leader
let slave_fd = open(Path::new(&slave_name), O_RDWR, stat::Mode::empty())?;
// assign stdin, stdout, stderr to the tty, just like a terminal does
dup2(slave_fd, STDIN_FILENO)?;
dup2(slave_fd, STDOUT_FILENO)?;
dup2(slave_fd, STDERR_FILENO)?;
std::process::Command::new("cat").status()?;
}
Ok(ForkResult::Parent { child: _ }) => {
let f = unsafe { std::fs::File::from_raw_fd(master_fd.as_raw_fd()) };
let mut reader = BufReader::new(&f);
let mut writer = LineWriter::new(&f);
writer.write_all(b"hello world\n")?;
let mut s = String::new();
reader.read_line(&mut s)?; // what we just wrote in
reader.read_line(&mut s)?; // what cat wrote out
writer.write(&[3])?; // send ^C
writer.flush()?;
let mut buf = [0; 2]; // needs bytewise read as ^C has no newline
reader.read(&mut buf)?;
s += &String::from_utf8_lossy(&buf).to_string();
println!("{}", s);
println!("cat exit code: {:?}", wait::wait()?); // make sure cat really exited
}
Err(_) => println!("error"),
}
Ok(())
}
fn main() {
run().expect("could not execute command");
}
Output:
hello world
hello world
^C
cat exit code: Signaled(2906, SIGINT, false)
Try adding -t option TWICE to force pseudo-tty allocation. I.e.
klar (16:14) ~>echo foo | ssh user#host.ssh.com tty
not a tty
klar (16:14) ~>echo foo | ssh -t -t user#host.ssh.com tty
/dev/pts/0
When you have a pseudo-tty, I think it should convert that to SIGINT as you wanted to do.
In your simple example, you could also just close stdin after the write, in which case the server should exit. For this particular case it would be more elegant and probably more reliable.
Solution without using a crate
Now that you are spawning a command in Rust, you might as well spawn another to send SIGINT to it. That command is kill.
So, you can do this:
use std::process::{Command, Stdio};
use std::io::{Result, Write};
fn main() -> Result<()> {
let mut child = Command::new("sh")
.arg("-c").arg("-i").arg("cat")
.stdin(Stdio::piped())
.spawn()?;
let mut stdin = child.stdin.take().unwrap();
let mut kill = Command::new("kill")
.arg(child.id().to_string())
.spawn()?;
kill.wait()
}
If I press Ctrl+C, this throws an exception (always in thread 0?). You can catch this if you want - or, more likely, run some cleanup and then rethrow it. But the usual result is to bring the program to a halt, one way or another.
Now suppose I use the Unix kill command. As I understand it, kill basically sends a (configurable) Unix signal to the specified process.
How does the Haskell RTS respond to this? Is it documented somewhere? I would imagine that sending SIGTERM would have the same effect as pressing Ctrl+C, but I don't know that for a fact...
(And, of course, you can use kill to send signals that have nothing to do with killing at all. Again, I would imagine that the RTS would ignore, say, SIGHUP or SIGPWR, but I don't know for sure.)
Googling "haskell catch sigterm" led me to System.Posix.Signals of the unix package, which has a rather nice looking system for catching and handling these signals. Just scroll down to the "Handling Signals" section.
EDIT: A trivial example:
import System.Posix.Signals
import Control.Concurrent (threadDelay)
import Control.Concurrent.MVar
termHandler :: MVar () -> Handler
termHandler v = CatchOnce $ do
putStrLn "Caught SIGTERM"
putMVar v ()
loop :: MVar () -> IO ()
loop v = do
putStrLn "Still running"
threadDelay 1000000
val <- tryTakeMVar v
case val of
Just _ -> putStrLn "Quitting" >> return ()
Nothing -> loop v
main = do
v <- newEmptyMVar
installHandler sigTERM (termHandler v) Nothing
loop v
Notice that I had to use an MVar to inform loop that it was time to quit. I tried using exitSuccess from System.Exit, but since termHandler executes in a thread that isn't the main one, it can't cause the program to exit. There might be an easier way to do it, but I've never used this module before so I don't know of one. I tested this on Ubuntu 12.10.
Searching for "signal" in the ghc source code on github revealed the installDefaultSignals function:
void
initDefaultHandlers(void)
{
struct sigaction action,oact;
// install the SIGINT handler
action.sa_handler = shutdown_handler;
sigemptyset(&action.sa_mask);
action.sa_flags = 0;
if (sigaction(SIGINT, &action, &oact) != 0) {
sysErrorBelch("warning: failed to install SIGINT handler");
}
#if defined(HAVE_SIGINTERRUPT)
siginterrupt(SIGINT, 1); // isn't this the default? --SDM
#endif
// install the SIGFPE handler
// In addition to handling SIGINT, also handle SIGFPE by ignoring it.
// Apparently IEEE requires floating-point exceptions to be ignored by
// default, but alpha-dec-osf3 doesn't seem to do so.
// Commented out by SDM 2/7/2002: this causes an infinite loop on
// some architectures when an integer division by zero occurs: we
// don't recover from the floating point exception, and the
// program just generates another one immediately.
#if 0
action.sa_handler = SIG_IGN;
sigemptyset(&action.sa_mask);
action.sa_flags = 0;
if (sigaction(SIGFPE, &action, &oact) != 0) {
sysErrorBelch("warning: failed to install SIGFPE handler");
}
#endif
#ifdef alpha_HOST_ARCH
ieee_set_fp_control(0);
#endif
// ignore SIGPIPE; see #1619
// actually, we use an empty signal handler rather than SIG_IGN,
// so that SIGPIPE gets reset to its default behaviour on exec.
action.sa_handler = empty_handler;
sigemptyset(&action.sa_mask);
action.sa_flags = 0;
if (sigaction(SIGPIPE, &action, &oact) != 0) {
sysErrorBelch("warning: failed to install SIGPIPE handler");
}
set_sigtstp_action(rtsTrue);
}
From that, you can see that GHC installs at least SIGINT and SIGPIPE handlers. I don't know if there are any other signal handlers hidden in the source code.