I have found usages of single functions such as T(), A() and D().
For example, this from the usql notebook https://legacy.gitbook.com/book/saveenr/usql-tutorial/details
#departments =
SELECT *
FROM (VALUES
(31, "Sales"),
(33, "Engineering"),
(34, "Clerical"),
(35, "Marketing")
) AS D( DepID, DepName );
Can someone please help me understand the meaning of these single letter functions or what they are?
U-SQL has the ability to create inline functions which are covered in great detail here:
https://sqlplayer.net/2017/10/functions-in-the-usql-the-hidden-gem-in-the-summer-2017-update/
You can do similar in Databricks, eg here's an example using Scala based on this U-SQL example:
import org.apache.spark.sql.functions.{col, expr, split, udf}
import scala.math._
// Define the function
val x = udf((s: Long, t: Long) => pow(s,t))
// Setup
val df = Seq( ( 1, "10^2" ), ( 2, "12^12" ), ( 3, "33^3") )
.toDF("rowId", "expo")
// Use the function
df
.withColumn( "col1", split( $"expo", "\\^").getItem(0) )
.withColumn( "col2", split( $"expo", "\\^").getItem(1) )
.withColumn( "col3", expr("pow(col1, col2)") )
.withColumn( "col4", x($"col1", $"col2") )
.show()
I would generally recommend longer function names however, in the interests of making code self-documenting.
Related
This question has already been asked on another platform, but I haven't got an answer yet.
https://discourse.julialang.org/t/generalizing-the-inputs-of-the-nlsolve-function-in-julia/
After an extensive process usyng the SymPy in Julia, I generated a system of nonlinear equations. My system is allocated in a matrix NxS. Something like this(NN = 2, S = 2).
I would like to adapt the system to use the NLsolve package. I do some boondoggle for the case NN=1 and S =1. The system_equations2 function give me the nonlinear system, like the figure
using SymPy
using Plots
using NLsolve
res = system_equations2()
In order to simulate the output, I do this:
NN = 1
S = 1
p= [Sym("p$i$j") for i in 1:NN,j in 1:S]
res = [ Eq( -331.330122303069*p[i,j]^(1.0) + p[i,j]^(2.81818181818182) - 1895.10478893046/(p[i,j]^(-1.0))^(2.0),0 ) for i in 1:NN,j in 1:S]
resf = convert( Function, lhs( res[1,1] ) )
plot(resf, 0 ,10731)
Now
resf = convert( Function, lhs( res[1,1] ) )
# This for the argument in the nlsolve function
function resf2(p)
p = Tuple(p)[1]
r = resf(p)
return r
end
Now, I find the zeros
function K(F,p)
F[1] = resf2(p[1])
end
nlsolve(K , [7500.8])
I would like to generalize this price to any NN and any S. I believe there is a simpler way to do this.
I'm new to drake but loving it so far. One thing I'm having trouble with is how to best go about experimenting with different pipeline configurations. That is, my plans consist purely of a chain of targets where the output from the first target is the input for the second, the second forms the input for the third, etc. My targets all have the same basic structure (dynamic targets with tibbles as individual entries) expected as input and supplied as output, and I want to experiment with different orderings, inclusion/exclusion of certain steps, etc. For example:
plan = drake::drake_plan(
a_transformed = target(
compute_a_transform(list_of_input_data)
, dynamic = map(list_of_input_data)
)
, b_transformed = target(
compute_b_transform(a_transformed)
, dynamic = map(a_transformed)
)
, c_transformed = target(
compute_c_transform(c_transformed)
, dynamic = map(c_transformed)
)
)
The way I've been using drake so far is that each target has a unique/meaningful name, so when I, for example, remove a target, I have to rename the input supplied to the subsequent target:
plan = drake::drake_plan(
a_transformed = target(
compute_a_transform(list_of_input_data)
, dynamic = map(list_of_input_data)
)
#, b_transformed = target(
# compute_b_transform(a_transformed)
# , dynamic = map(a_transformed)
#)
#note the b-transform step has been removed (commented-out), requiring inputs to c_transform to be changed from `b_transform` to `a_transform`
, c_transformed = target(
compute_c_transform(a_transformed) #had to rename things here
, dynamic = map(a_transformed) #and here
)
)
Would it be too much to hope that there's a better way of experimenting that doesn't require this manual commenting-out and renaming?
I worked out a method that is a bit of a hack but works for me. I simply add a skip argument to each function that triggers return of the input if TRUE:
compute_a_transform = function(x,skip=F){
if(skip){
return(x)
}
... #regular compute_a_transform stuff here
}
Then, when I want to skip a step in the processing chain, I simply set skip=TRUE without commenting-out or renaming anything
plan = drake::drake_plan(
a_transformed = target(
compute_a_transform(list_of_input_data)
, dynamic = map(list_of_input_data)
)
, b_transformed = target(
compute_b_transform(a_transformed, skip=TRUE) #skip=TRUE means the b-transform isn't actually applied
, dynamic = map(a_transformed)
)
, c_transformed = target(
compute_c_transform(c_transformed)
, dynamic = map(c_transformed)
)
)
In the Help Documentation of Scilab 6.0.2, I can read the following instruction on the Overloading entry, regarding the last operation code "iext" showed in this entry's table:
"The 6 char code may be used for some complex insertion algorithm like x.b(2) = 33 where b field is not defined in the structure x. The insertion is automatically decomposed into temp = x.b; temp(2) = 33; x.b = temp. The 6 char code is used for the first step of this algorithm. The 6 overloading function is very similar to the e's one."
But I can't find a complete example on how to use this "char 6 code" to overload a function. I'm trying to use it, without success. Does anyone have an example on how to do this?
The code bellow creates a normal "mlist" as a example. Which needs overloading functions
A = rand(5,3)
names = ["colA" "colB" "colC"]
units = ["ft" "in" "lb"]
M = mlist(["Mlog" "names" "units" names],names,units,A(:,1),A(:,2),A(:,3))
Following are the overload functions:
//define display
function %Mlog_p(M)
n = size(M.names,"*")
formatStr = strcat(repmat("%10s ",1,n)) + "\n"
formatNum = strcat(repmat("%0.10f ",1,n)) + "\n"
mprintf(formatStr,M.names)
mprintf(formatStr,M.units)
disp([M(M.names(1)),M(M.names(2)),M(M.names(3))])
end
//define extraction operation
function [Mat]=%Mlog_e(varargin)
M = varargin($)
cols = [1:size(M.names,"*")] // This will also work
cols = cols(varargin($-1)) // when varargin($-1) = 1:1:$
Mat = []
if length(varargin)==3 then
for i = M.names(cols)
Mat = [Mat M(i)(varargin(1))]
end
else
for i=1:size(M.names(cols),"*")
Mat(i).name = M.names(cols(i))
Mat(i).unit = M.units(cols(i))
Mat(i).data = M(:,cols(i))
end
end
endfunction
//define insertion operations (a regular matrix into a Mlog matrix)
function ML=%s_i_Mlog(i,j,V,M)
names = M.names
units = M.units
A = M(:,:) // uses function above
A(i,j) = V
ML = mlist(["Mlog" "names" "units" names],names,units,A(:,1),A(:,2),A(:,3))
endfunction
//insertion operation with structures (the subject of the question)
function temp = %Mlog_6(j,M)
temp = M(j) // uses function %Mlog_e
endfunction
function M = %st_i_Mlog(j,st,M)
A = M(:,:) // uses function %Mlog_e
M.names(j) = st.name // uses function above
M.units(j) = st.unit // uses function above
A(:,j) = st.data // uses function above
names = M.names
units = M.units
M = mlist(["Mlog" "names" "units" names],names,units,A(:,1),A(:,2),A(:,3))
endfunction
The first overload (displays mlist) will show the matrix in the form of the following table:
--> M
M =
colA colB colC
ft in lb
0.4720517 0.6719395 0.5628382
0.0623731 0.1360619 0.5531093
0.0854401 0.2119744 0.0768984
0.0134564 0.4015942 0.5360758
0.3543002 0.4036219 0.0900212
The next overloads (extraction and insertion) Will allow the table to be access as a simple matrix M(i,j).
The extraction function Will also allow M to be access by column, which returns a structure, for instance:
--> M(2)
ans =
name: "colB"
unit: "in"
data: [5x1 constant]
The last two functions are the overloads mentioned in the question. They allow the column metadata to be changed in a structure form.
--> M(2).name = "length"
M =
colA length colC
ft in lb
0.4720517 0.6719395 0.5628382
0.0623731 0.1360619 0.5531093
0.0854401 0.2119744 0.0768984
0.0134564 0.4015942 0.5360758
0.3543002 0.4036219 0.0900212
I'm looking for a straight forward way to change the formatting of numbers into K,M in shiny dataTables. Preferably with something like formatCurrency. I don't want to write k, m functions to convert number into string in order to do the formatting as it makes it difficult to sort rows by value.
There's no built-in way to do this, but it's not too bad to write your own format function in JavaScript that doesn't break row sorting.
See Column Rendering in the DT docs for how to do this: https://rstudio.github.io/DT/options.html
And this will also help:
https://datatables.net/reference/option/columns.render
Here's an example of a custom thousands formatter that rounds to 1 decimal place:
library(DT)
formatThousands <- JS(
"function(data) {",
"return (data / 1000).toFixed(1) + 'K'",
"}")
datatable(datasets::rock, rownames = FALSE, options = list(
columnDefs = list(list(
targets = 0:1, render = formatThousands
))
))
Alternatively, if you want a non-JavaScript method, you could use the colFormat function used with the reactable package. Unfortunately, there is no automatic millions option but it's pretty easy to replicate if you divide the original data and add the labels on with colFormat.
Product <- c('Apples','Oranges','Pears')
Revenue <- c(212384903, 23438872, 26443879)
df <- data.frame(Product,Revenue)
df$Revenue_millions <- dfeg$Revenue/1000000
reactable(df,
showSortable = TRUE,
columns = list(
Revenue_millions = colDef(format = colFormat(prefix = "£", separators = TRUE,digits=1,suffix = "m"))))
The data should now sort correctly
If you are using DataTables, to get the data as Unit format i.e
10000 -> 10K
we can use render function
"render": function ( data ) {
if(data > 999 && data < 1000000) {
return data/1000+' K'
}
else if(data > 1000000){
return data/1000000+' M'
}
else{
return data
}
}
}
I'm a huge fan of S4 Reference Classes as they allow for a hybrid programming style (functional/pass-by-value vs. oop/pass-by-reference; example) and thus increase flexibility dramatically.
However, I think I just came across an undesired behavior with respect to the way R scans through environments/frames when you ask it to retrieve a certain field value via method $field() (see help page). The problem is that R also seems to look in enclosing environments/frames if the desired field is not found in the actual local/target environment (which would be the environment making up the S4 Reference Class), i.e. it's just like running get(<objname>, inherits=TRUE) (see help page).
Actual question
In order to have R just look in the local/target environment, I was thinking something like $field(name="<fieldname>", inherits=FALSE) but $field() doesn't have a ... argument that would allow me to pass inherits=FALSE along to get() (which I'm guessing is called somewhere along the way). Is there a workaround to this?
Code Example
For those interested in more details: here's a little code example illustrating the behavior
setRefClass("A", fields=list(a="character"))
x <- getRefClass("A")$new(a="a")
There is a field a in class A, so it's found in the target environment and the value is returned:
> x$field("a")
[1] "a"
Things look differently if we try to access a field that is not a field of the reference class but happens to have a name identical to that of some other object in the workspace/searchpath (in this case "lm"):
require("MASS")
> x$field("lm")
function (formula, data, subset, weights, na.action, method = "qr",
model = TRUE, x = FALSE, y = FALSE, qr = TRUE, singular.ok = TRUE,
contrasts = NULL, offset, ...)
{
ret.x <- x
ret.y <- y
[omitted]
if (!qr)
z$qr <- NULL
z
}
<bytecode: 0x02e6b654>
<environment: namespace:stats>
Not really what I would expect at this point. IMHO an error or at least a warning would be much better. Or opening method $field() for arguments that can be passed along to other functions via .... I'm guessing somewhere along the way get() is called when calling $field(), so something like this could prevent the above behavior from occurring:
x$field("digest", inherits=FALSE)
Workaround: own proposal
This should do the trick, but maybe there's something more elegant that doesn't involve the specification of a new method on top of $field():
setRefClass("A", fields=list(a="character"),
methods=list(
myField=function(name, ...) {
# VALIDATE NAME //
if (!name %in% names(getRefClass(class(.self))$fields())) {
stop(paste0("Invalid field name: '", name, "'"))
}
# //
.self$field(name=name)
}
)
)
x <- getRefClass("A")$new(a="a")
> x$myField("a")
[1] "a"
> x$myField("lm")
Error in x$myField("lm") : Invalid field name: 'lm'
The default field() method can be replaced with your own. So adding an inherits argument to avoid the enclosing frames is simply a matter of grabbing the existing x$field definition and adding it...
setRefClass( Class="B",
fields= list( a="character" ),
methods= list(
field = function(name, value, inherits=TRUE ) {
if( missing(value) ) {
get( name, envir=.self, inherits=inherits )
} else {
if( is.na( match( name, names( .refClassDef#fieldClasses ) ) ) ) {
stop(gettextf("%s is not a field in this class", sQuote(name)), domain = NA)
}
assign(name, value, envir = .self)
}
}
),
)
Or you could have a nice error message with a little rearranging
setRefClass( Class="C",
fields= list( a="character" ),
methods= list(
field = function(name, value, inherits=TRUE ) {
if( is.na( match( name, names( .refClassDef#fieldClasses ) ) ) &&
( !missing(value) || inherits==FALSE) ) {
stop(gettextf("%s is not a field in this class", sQuote(name)), domain = NA)
}
if( missing(value) ) {
get( name, envir=.self, inherits=inherits )
} else {
assign(name, value, envir = .self)
}
}
),
)
Since you can define any of your own methods to replace the defaults pretty much any logic you want can be implemented for your refclasses. Perhaps an error if the variable is acquired using inheritance but the mode matches to c("expression", "name", "symbol", "function") and warning if it doesn't directly match the local refClass field names?