I'm exploring these two libs and for me ImmutableJS has (mostly) immutable data structures whereas Ramda has swiss army knife set of FP utils.
When I google I see articles like Ramda vs ImmutableJS that recommend one or the other, but at first glance, at least to me, they complement each other.
I can imagine having all my data structures in React app to be ImmutableJS (List, Set, Map, Record) but use Ramda to compose functions, make transformations on ImmutableJS collections etc or when I miss some features like uniq.
What do you think? Is it sensible approach?
Can they be used together; not smoothly or seamlessly.
It is most probably an approach that, as things stand currently, will lead you toward at least some difficulty, and most probably a great deal of difficulty. The reason for this is that immutable's structures are not compatible with ramda's functions. You may be able to create wrappers or an interop to patch this functionality in, but this seems like a fairly big project.
This issue goes into further detail, and includes some potential alternatives to immutablejs such as List.
Here's a very small example of why the lack of interoperability is a problem:
const { Map } = Immutable;
const { path } = R;
const standardObject = {a: 1, b: 2, c: 3};
const map1 = Immutable.Map(standardObject);
const getA = path(['a']);
const result = getA(map1);
const defaultResult = getA(standardObject);
console.dir(result);
console.dir(defaultResult);
<script src="https://cdnjs.cloudflare.com/ajax/libs/ramda/0.26.1/ramda.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/immutable/3.8.2/immutable.js"></script>
You could presumably build the conversions into the work you're doing in ramda; this seems like the simplest way around the problem though it does mean you'd be tying your code very closely to immutable data structures. That might work like this:
const { Map } = Immutable;
const { path, pipe } = R;
const standardObject = {a: 1, b: 2, c: 3};
const map1 = Immutable.Map(standardObject);
const toStandard = map => map.toObject()
const getA = pipe(
toStandard,
path(['a'])
)
const result = getA(map1)
console.dir(result)
<script src="https://cdnjs.cloudflare.com/ajax/libs/ramda/0.26.1/ramda.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/immutable/3.8.2/immutable.js"></script>
Some of this integration comes free with Ramda. Ramda delegates many of its calls to the data object. Mostly, if the name of a Ramda function is the same as the name of an Immutable method, then Ramda will perform the integration.
This means that you can often use an Immutable List or Map with Ramda where you would usually use an Array or Object.
const {Map, List} = immutable
const {map, filter, find} = ramda
const square = n => n * n
const isEven = n => n % 2 == 0
const xs = new List([1, 2, 3, 4, 5])
console.log(map(square, xs)) //=> List [1, 4, 9, 16, 25]
console.log(filter(isEven, xs)) //=> List [2, 4]
console.log(find(n => n > 3, xs)) //=> 3
const x = new Map({a: 1, b: 2, c: 3, d: 4, e: 5});
console.log(map(square, x))
//=> Map {"a" => 1, "b" => 4, "c" => 9, "d" => 16, "e" => 25}
console.log(filter(isEven, x)) //=> Map {"c" => 3, "d" => 4, "e" => 5}
<script src="https://bundle.run/ramda#0.26.1"></script>
<script src="https://bundle.run/immutable#4.0.0-rc.12"></script>
This is limited, though, for now by the shared names. And not all of Ramda's functions delegate this way.
As the issue mentioned by Oliver Radini describes, I have some real interest in making it easier in Ramda to do further integrations with tools like Immutable, but it's never made it to the top of the list, and there is some real pushback among the Ramda contributors.
Related
Associativity is a desirable property and quite common for many operations in FP. Now I was wondering if an impure function can interfere with it. The only example I found isn't really convincing, because I'm not sure if nullary functions count as proper functions (riegorously speaking) and additionally, the example looks rather contrived.
The following is written in JS but is hopefully self-explanatory:
// function composition
const comp = f => g => x => f(g(x));
const foo = () => 1;
const bar = () => Math.round(Math.random() * 100);
// Set functor (ignore the hideous implementation)
const map = f => s => {
const r = new Set();
s.forEach(x => r.add(f(x)));
return r;
};
const lhs = map(comp(bar) (foo));
const rhs = comp(map(bar)) (map(foo));
const set1 = lhs(new Set([1, 2, 3]));
const set2 = rhs(new Set([1, 2, 3]));
console.log(Array.from(set1)); // yields an array filled with up to three random integers
console.log(Array.from(set2)); // yields an array filled with a single random integer
I'm not sure whether this example can be considered as evidence. Are there more convincing examples?
I am still new to functional programming and have been trying to learn how to use transducers. I thought I had a good use case but every time I attempt to write a transducer with Ramda for it, I get the following error:
reduce: list must be array or iterable
I have tried rewriting it several ways and looked at several explanations on the web of transduction but to no avail. Any suggestions?
const data = [{cost:2,quantity:3},{cost:4,quantity:5},{cost:1,quantity:1}];
const transducer = R.compose(R.map(R.product), R.map(R.props(['cost', 'quantity'])));
const result = R.transduce(transducer, R.add, 0)(data);
console.log(result)
In the context of a transducer, compose reads left to right. You just need to invert product and props:
const data = [
{cost:2,quantity:3},
{cost:4,quantity:5},
{cost:1,quantity:1}];
const transducer =
compose(
map(props(['cost', 'quantity'])),
map(product));
console.log(
transduce(transducer, add, 0, data)
)
<script src="https://cdnjs.cloudflare.com/ajax/libs/ramda/0.26.1/ramda.min.js"></script>
<script>const {compose, map, props, product, transduce, add} = R;</script>
The reason why the order reverses is that transducers utilize a property of function composition that is sometimes called abstraction from arity. It simply means that a function composition can return, well, another function:
const comp = f => g => x => f(g(x));
const mapTrace = tag => f => (console.log(tag), xs => (console.log(tag), xs.map(f)));
const sqr = x => x * x;
const main = comp(mapTrace("a")) (mapTrace("b")) (sqr); // returns another function
console.log(main); // logs the 2nd map and then the 1st one (normal order)
// pass an additional argument to that function
console.log(
main([[1,2,3]])); // logs in reverse order
Why returns the composition another function? Because map is a binary function that expects a function argument as its first argument. So when the composition is evaluated it yields another compositon of two partially applied maps. It is this additional iteration that reverses the order. I stop at this point without illustrating the evaluation steps, because I think it would get too complicated otherwise.
Additionally, you can see now how transducers fuse two iterations together: They simply use function composition. Can you do this by hand? Yes, you can absolutely do that.
I'm trying to remove some elements from a vector, based on a predicate, and collecting the result. Here's a (not working) example with an expected result:
let mut v: Vec<i32> = vec![1, 2, 3, 4, 5, 6];
let drained: Vec<i32> = v.iter().filter(|e| (*e) % 2 == 0).drain(..).collect();
assert_eq!(v, vec![1, 3, 5]);
assert_eq!(drained, vec![2, 4, 6]);
This results in the error
error[E0599]: no method named `drain` found for type `std::iter::Filter<std::slice::Iter<'_, i32>, [closure#src/main.rs:4:45: 4:62]>` in the current scope
--> src/main.rs:4:64
|
4 | let drained: Vec<i32> = v.iter().filter(|e| (*e) % 2 == 0).drain(..).collect();
| ^^^^^
There are several alternatives I looked at, none of them seem to be doing what I want:
Vec::retain removes the elements from the vector, but doesn't give back ownership of the removed elements.
v.drain(..).filter(condition).collect() returns the correct value for drained but empties the whole vector.
Not in stable Rust 1.33.0. There's an unstable nightly feature called drain_filter that does exactly what you want:
#![feature(drain_filter)]
fn main() {
let mut v: Vec<i32> = vec![1, 2, 3, 4, 5, 6];
let drained: Vec<i32> = v.drain_filter(|&mut e| e % 2 == 0).collect();
assert_eq!(v, vec![1, 3, 5]);
assert_eq!(drained, vec![2, 4, 6]);
}
As a stable workaround, you may be able to use Iterator::partition, but it does not reuse the memory:
fn main() {
let v: Vec<i32> = vec![1, 2, 3, 4, 5, 6];
let (drained, v): (Vec<_>, Vec<_>) = v.into_iter().partition(|&e| e % 2 == 0);
assert_eq!(v, vec![1, 3, 5]);
assert_eq!(drained, vec![2, 4, 6]);
}
The documentation state that Vec.retain will operate in-place, and visit each element in order, exactly once.
fn drain_where<T, Pred : Fn(&T) -> bool>(source: &mut Vec<T>, pred: Pred) -> Vec<T>
where T : Copy {
let mut drained: Vec<T> = Vec::new();
source.retain(|item| {
if pred(item) { drained.push(*item); false } else { true }
});
drained
}
I can suggest few other ways to do this + my benchmarks.
N.B. I compare all methods by few criteria:
Does it support external source of truth (my use-case). E.g. Vec::retain supports that, meaning that you can write code like
// conditions: &[bool]
assert_eq!(conditions.len(), my_vec.len());
let cond = conditions.iter().copied();
my_vec.retain(move|_|cond.next().unwrap());
Is method supported by third-party Vecs, namely ArrayVec, TinyVec, SmallVec, FixedSliceVec and others.
Is it fast.
So, let's begin
Sort slice than split slice
Features:
Can support external source of truth — No ❌. Would call closure O(n log n) times in unspecified order so support only predicates which calculated only directly from values.
Third-party support — Excellent ✅. You can use it on anything convertible to mutable slice.
Is it fast — In generic case, no ❌. It runs for O(n log n) time while other methods run for O(n) time. However, if preserving original relative order is not important for you, you can use sort_unstable_by_key which doesn't allocate memory at all, which can make it fastest in some scenarios.
Implementation:
v.sort_by_key(|x| predicate(x));
let split_pos = v.partition_point(|x| predicate(x));
let (false_slice, true_slice) = v.split_at_mut(split_pos)
Vec::drain_filter
Can support external source of truth — Yes ✅. Visits items in original order exactly one time.
Third-party support — Non existent ❌. Also, you can't it even use in stable Rust and it's tracking issue has been suffering from bikeshedding 5 years now (at 2022-07).
Is it fast — Yes ✅.
Code
let removed_items: Vec<_> = v.drain_filter(|x| predicate(x)).collect();
MaybeUninit trick using unsafe code
Well, I wrote it myself.
Features:
Can support external source of truth — Yes ✅. Visits items in original order exactly one time.
Third-party support — Supported ✅. Note that you must audit their implementation of retain to ensure that it cannot panic itself.
Is it fast — Yes ✅. In my benchmarks it is faster than Vec::drain_filter.
This implementation makes 2 assumptions:
Internal layout of Vec<T> and Vec<MaybeUninit<T>> is same. There is no reason why it is not because memory layout of T and MaybeUninit<T> is same but I also verify this using asserts. Note that asserts would be removed by compiler optimizer because they are always true.
retain doesn't panic itself (it can only propagate panics from element drop or from predicate). This is true for std::vec::Vec but you need to ensure that for third-party crates.
Algorithm is simple:
reinterpret initial vector Vec<T> as Vec<MaybeUninit<T>>;
wrap our predicate into new predicate that moves items which we want to remove into external storage;
let Vec::retain handle removal of items.
Also, only reason of usage of MaybeUninit and unsafe is avoiding double-frees so if your elements implement Copy, this algorithm can be implemented in safe Rust.
However, in that case you can just use filter(...).collect() and retain with almost same performance.
So, code is below with all comments why it is safe (note that I didn't test it using sanitizers or Miri so use it on your own risk):
/// Returns removed values.
fn retain_unsafe_generic<T: Sized>(
v: &mut Vec<T>,
mut which_to_keep: impl FnMut(&T) -> bool,
) -> Vec<T> {
use std::mem::{transmute, MaybeUninit};
/// # Safety
/// Caller must ensure that if it makes living copies of inner items,
/// those items is removed from original vec before original reference became usable again.
unsafe fn as_uninits<T: Sized>(v: &mut Vec<T>) -> &mut Vec<MaybeUninit<T>> {
let orig_ptr = v.as_ptr();
let orig_cap = v.capacity();
let orig_size = v.len();
let v: &mut Vec<MaybeUninit<T>> = unsafe {
// Safety: since `MaybeUninit` has same memory layout
// as wrapped type, we assume that we can treat vec with T
// as MaybeUninit<T>. This assumption checked by asserts below.
//
// Lifetimes of elements must be correctly enforced by caller.
transmute(v)
};
// Check if layout of Vec with different element type remains same.
assert_eq!(v.len(), orig_size);
assert_eq!(v.capacity(), orig_cap);
assert_eq!(v.as_ptr(), orig_ptr.cast());
v
}
let mut res: Vec<T> = Vec::with_capacity(v.len());
let v = unsafe {
// Safety: we keep result reference only in `retain` call.
// We would remove all moved elements using retain.
as_uninits(v)
};
v.retain(
// Safety: `Vec::retain` would remove all items which values we moved into `res`.
// It wouldn call `drop::<T>` for removed values
// because `MaybeUninit` never drops wrapped values.
|x| unsafe {
// Safety: it is safe because `Vec::retain` visits elements sequentally
// so we haven't moved value from `x` yet.
// https://doc.rust-lang.org/std/vec/struct.Vec.html#method.retain
let val = &*x.as_ptr();
if which_to_keep(val) {
return true;
}
res.reserve(1);
// Any panic before this place is safe because
// 1. We didn't moved value from `x` yet;
// 2. In case of panic in predicate, `Vec::retain` preserve current value.
// Here we could probably use `Vec::push`
// but compiler currently fails to remove capacity check in `Vec::push`
// so this function became slower than `Vec::drain_filter`
// https://godbolt.org/z/7fhnnMh46
// And `Vec::push(x.assume_init_read())` is unsafe operation too anyway.
let old_len = res.len();
// Safety: We just allocated memory for this place.
let dst = res.as_mut_ptr().add(old_len);
// Safety: since we cannot panic until the end of closure
// and `Vec::retain` wouldn't panic and would remove `x`,
// making bitwise copy of `x` is safe.
x.as_ptr().copy_to_nonoverlapping(dst, 1);
// Safety: we just wrote additional value.
res.set_len(old_len + 1);
false
},
);
res
}
Benchmarks
Code of benchmarks is long so here link to the gist: https://gist.github.com/AngelicosPhosphoros/7ee482316bc1c83945f88308954e0d7e
It tries to split away odd numbers from they Vec using all three algorithms I listed.
Results:
algorithm
Mixed
Odds first
Evens first
sort-split
465us
35us
10us
drain_filter
26us
24us
22.5us
retain-uninit
17us
21us
19us
As you see, retain usage won in all cases except when sort-split doesn't actually have anything to do.
It is mainly because Vec::retain has been rigorously optimized over the years.
Here are four functions I am trying to compose into a single endpoint string:
const endpoint = str => `${str}` || 'default'
const protocol = str => `https://${str}`
const params = str => `${str}?sort=desc&part=true&`
const query = str => `${str}query={ some:'value', another:'value'}`
let finalEndpoint = R.compose(query, params, protocol, endpoint)
var result = finalEndpoint('api.content.io')
This composition works and returns the result I want which is:
https://api.content.io?sort=desc&part=true&query={ some:'value', another:'value'}
But notice how I have hard coded the values for params and query inside their function body. I see only one value going up the value in this R.compose chain.
How and where exactly do I pass in parameters to the params and query parameters?
UPDATE:
What I did was curried those functions like this:
var R = require('ramda');
const endpoint = str => `${str}` || 'default'
const protocol = str => `https://${str}`
const setParams = R.curry ( (str, params) => `${str}?${params}` )
const setQuery = R.curry ( (str, query) => `${str}&query=${JSON.stringify(query)}` )
and then
let finalEndpoint = R.compose(protocol, endpoint)
var result = setQuery(setParams(finalEndpoint('api.content.io'), 'sort=desc&part=true'), { some:'value', another:'value'})
console.log(result);
But the final call to get result still seems pretty hacked and inelegant. Is there any way to improve this?
How and where exactly do I pass in parameters to the params and query parameters?
Honestly, you don't, not when you're building a compose or pipe pipeline with Ramda or similar libraries.
Ramda (disclaimer: I'm one of the authors) allows the first function to receive multiple arguments -- some other libraries do, some don't -- but subsequent ones will only receive the result of the previous calls. There is one function in Sanctuary, meld, which might be helpful with this, but it does have a fairly complex API.
However, I don't really understand why you are building this function in this manner in the first place. Are those intermediate functions actually reusable, or are you building them on spec? The reason I ask is that this seems a more sensible version of the same idea:
const finalEndpoint = useWith(
(endpoint, params, query) =>`https://${endpoint}?${params}&query=${query}`, [
endpoint => endpoint || 'default',
pipe(toPairs, map(join('=')), join('&')),
pipe(JSON.stringify, encodeURIComponent)
]
);
finalEndpoint(
'api.content.io',
{sort: 'desc', part: true},
{some:'value', another:'value'}
);
//=> "https://api.content.io?sort=desc&part=true&query=%7B%22some%22%3A%22value%22%2C%22another%22%3A%22value%22%7D"
I don't really know your requirements for that last parameter. It looked strange to me without that encodeUriComponent, but perhaps you don't need it. And I also took liberties with the second parameter, assuming that you would prefer actual data in the API to a string encapsulating that data. But if you want to pass 'sort=desc&part=true', then replace pipe(toPairs, map(join('=')), join('&')) with identity.
Since the whole thing is far from points-free, I did not use a points-free version of the first function, perhaps or(__, 'default'), as I think what's there is more readable.
Update
You can see a version of this on the Ramda REPL, one that adds some console.log statements with tap.
This does raise an interesting question for Ramda. If those intermediate functions really are desirable, Ramda offers no way to combine them. Obviously Ramda could offer something like meld, but is there a middle ground? I'm wondering if there is a useful function (curried, of course) that we should include that works something like
someFunc([f0], [a0]); //=> f0(a0)
someFunc([f0, f1], [a0, a1]); //=> f1(f0(a0), a1)
someFunc([f0, f1, f2], [a0, a1, a2]); //=> f2(f1(f0(a0), a1), a2)
someFunc([f0, f1, f2, f3], [a0, a1, a2, a3]); //=> f3(f2(f1(f0(a0), a1), a2), a3)
// ...
There are some serious objections: What if the lists are of different lengths? Why is the initial call unary, and should we fix that by adding a separate accumulator parameter to the function? Nonetheless, this is an intriguing function, and I will probably raise it for discussion on the Ramda boards.
I wrote a little helper function for situations like this.
It is like compose, but with the rest params also passed in. The first param is the return value of the previous function. The rest params remain unchanged.
With it, you could rewrite your code as follows:
const compound = require('compound-util')
const endpoint = str => `${str}` || 'default'
const protocol = str => `https://${str}`
const params = (str, { params }) => `${str}?${params}`
const query = (str, { query }) => `${str}query=${query}`
const finalEndpoint = compound(query, params, protocol, endpoint)
const result = finalEndpoint('api.content.io', {
params: 'sort=desc&part=true&',
query: JSON.stringify({ some:'value', another:'value'})
})
If you have params and query as curried functions then you can:
EDIT: code with all the bells and whistles, needed to change parameter order or use R.__ and stringify object
const endpoint = R.curry( str => `${str}` || 'default' )
const protocol = R.curry( str => `https://${str}` )
const params = R.curry( (p, str) => `${str}?${p}` )
const query = R.curry( (q, str) => `${str}&query=${q}` )
let finalEndpoint =
R.compose(
query(JSON.stringify({ some:'value', another:'value' })),
params('sort=desc&part=true'),
protocol,
endpoint
)
var result = finalEndpoint('api.content.io')
console.log(result)
I have been practicing java 8 streams and functional style for a while.
Sometimes I try to solve some programming puzzles just using streams.
And during this time I found a class of tasks which I don't know how to solve with streams, only with classical approach.
One example of this kind of tasks is:
Given an array of numbers find index of the element which will make sum of the left part of array below zero.
e.g. for array [1, 2, 3, -1, 3, -10, 9] answer will be 5
My first idea was to use IntStream.generate(0, arr.length)... but then I don't know how to accumulate values and being aware of index same time.
So questions are:
Is it possible to somehow accumulate value over stream and then make conditional exit?
What is then with parallel execution? it's not fitting to problem of finding indexes where we need to be aware of elements order.
I doubt your task is well suited for streams. What you are looking for is a typical scan left operation which is by nature a sequential operation.
For instance imagine the following elements in the pipeline: [1, 2, -4, 5]. A parallel execution may split it into two subparts namely [1, 2] and [-4, 5]. Then what would you do with them ? You cannot sum them independently because it will yields [3] and [1] and then you lost the fact that 1 + 2 - 4 < 0 was respected.
So even if you write a collector that keeps track of the index, and the sum, it won't be able to perform well in parallel (I doubt you can even benefit from it) but you can imagine such a collector for sequential use :
public static Collector<Integer, ?, Integer> indexSumLeft(int limit) {
return Collector.of(
() -> new int[]{-1, 0, 0},
(arr, elem) -> {
if(arr[2] == 0) {
arr[1] += elem;
arr[0]++;
}
if(arr[1] < limit) {
arr[2] = 1;
}
},
(arr1, arr2) -> {throw new UnsupportedOperationException("Cannot run in parallel");},
arr -> arr[0]
);
}
and a simple usage:
int index = IntStream.of(arr).boxed().collect(indexSumLeft(0));
This will still traverse all the elements of the pipeline, so not very efficient.
Also you might consider using Arrays.parallelPrefix if the data-source is an array. Just compute the partial sums over it and then use a stream to find the first index where the sum is below the limit.
Arrays.parallelPrefix(arr, Integer::sum);
int index = IntStream.range(0, arr.length)
.filter(i -> arr[i] < limit)
.findFirst()
.orElse(-1);
Here also all the partial sums are computed (but in parallel).
In short, I would use a simple for-loop.
I can propose a solution using my StreamEx library (which provides additional functions to the Stream API), but I would not be very happy with such solution:
int[] input = {1, 2, 3, -1, 3, -10, 9};
System.out.println(IntStreamEx.of(
IntStreamEx.of(input).scanLeft(Integer::sum)).indexOf(x -> x < 0));
// prints OptionalLong[5]
It uses IntStreamEx.scanLeft operation to compute the array of prefix sums, then searches over this array using IntStreamEx.indexOf operation. While indexOf is short-circuiting, the scanLeft operation will process the whole input and create an intermediate array of the same length as the input which is completely unnecessary when solving the same problem in imperative style.
With new headTail method in my StreamEx library it's possibly to create lazy solution which works well for very long or infinite streams. First, we can define a new intermediate scanLeft operation:
public static <T> StreamEx<T> scanLeft(StreamEx<T> input, BinaryOperator<T> operator) {
return input.headTail((head, tail) ->
scanLeft(tail.mapFirst(cur -> operator.apply(head, cur)), operator)
.prepend(head));
}
This defines a lazy scanLeft using the headTail: it applies given function to the head and the first element of the tail stream, then prepends the head. Now you can use this scanLeft:
scanLeft(StreamEx.of(1, 2, 3, -1, 3, -10, 9), Integer::sum).indexOf(x -> x < 0);
The same can be applied to the infinite stream (e.g. stream of random numbers):
StreamEx<Integer> ints = IntStreamEx.of(new Random(), -100, 100)
.peek(System.out::println).boxed();
int idx = scanLeft(ints, Integer::sum).indexOf(x -> x < 0);
This will run till the cumulative sum becomes negative and returns the index of the corresponding element.