Learning more about how to write a query in Kusto. I have a column in 2 tables that have different Roles, but the column header is Role, that I'd like to combine the data into one column called Roles.
I tried, adding this, | extend Roles = strcat (RoleName, Role), but that just combined the data.
Here is my query attempt, I'm joining 3 tables, 2 of which have the roles. The third is where I'm validating the user aliases.
(cluster('****').database('****').****_****** | where Discriminator == 'Service'| where DivisionOid == '******')
| join kind = leftouter cluster('****').database('****').Release_Users on SubscriptionId
| join kind = leftouter (cluster('****').database('****').Release_AzureAccess
| where RoleId contains "****" and PrincipalType !contains "ServicePrincipal") on SubscriptionId
| join kind = leftouter cluster('****').database('****').Headtrax_PeopleHierarchyV1 on $left.PrincipalName == $right.EmailAddress and $left.LiveEmailId == $right.EmailAddress
| extend Roles = strcat (RoleName, Role)<<--this was my failed attempt at combining the Role columns. That just concatenated.
I want to validate each user is active from 2 different tables against a person table. I'm a novice and am struggling with how to get this right. I'm thinking I want to combine the 2 tables into one list rather than trying to combine one column out of the 2 tables. Anyone have any advice?
This seems like a job for the union operator.
Union takes two or more tables and returns the rows of all of them.
From: https://learn.microsoft.com/en-us/azure/kusto/query/unionoperator?pivots=azuredataexplorer
Here's an example query from the docs above -
let View_1 = view () { print x=1 };
let View_2 = view () { print x=toint(2) };
let View_3 = view () { print x_long=3 };
union withsource=TableName View_1, View_2, View_3
produces:
The answer from Irwin will certainly work. If you want to take his solution and flatten it into one row, you could do that with the summarize function.
let View_1 = view () { print x=1 };
let View_2 = view () { print x=toint(2) };
let View_3 = view () { print x_long=3 };
union withsource=TableName View_1, View_2, View_3
| summarize sum(x_long1), sum(x_int), sum(x_long)
Related
I am trying to access function parameters within the 'case' statement in that function and displaying data/"filtered" based on the permission flag..Is it possible?
Usecase: TypeCast the value based on the columnType and check if the user has the permission to view the column based on which you display either the value or say something like "filtered"
Here is what I tried
function rls_columnCheck
.create-or-alter function rls_columnCheck(tableName:string, columnName: string, value:string, columnType:string, IsInGroupPII:bool, IsInGroupFinance:bool) {
let PIIColumns = rls_getTablePermissions(tableName, "PII");
let FinanceColumns = rls_getTablePermissions(tableName, "Finance");
let val= case(columnType=="bool", tobool(value),
columnType=="datetime", todatetime(value),
columnType=="int", toint(value),
value);
iif(columnName in (PIIColumns),
iif(columnName in (FinanceColumns),
iif(IsInGroupPII == true and IsInGroupFinance == true,
val,
"filtered"), // PII True, Fin True
iif(IsInGroupPII == true,
val,
"filtered") // PII True, Fin False
),
iif(columnName in (FinanceColumns),
iif(IsInGroupFinance == true,
val,
"filtered"), // PII False, Fin True
val // PII False, Fin False
)
);
}
Error:
Call to iff(): #then data type (int) must match the #else data type (string)
val in your function must have a single and well-defined data type, that is known at "compile" time of the query.
you can't have different cases, where in each it has a different type (bool, datetime, int, string - in your case statement) - hence the error.
if it makes sense in your use case, you can try to always have val typed as string.
This is not a good approach to use RLS because this will actually cause the engine to run a function for every column of every record. It has many downsides:
Performance of displaying the table’s contents (even if you have full permissions)
Queries on the table won’t benefit from the indexes Kusto stores (suppose you query PermissionTesting2 | where Col1 has “blablabla” - instead of checking the index for “blablabla”, the engine will have to scan all the data, because it has to apply a function for every single cell)
A better approach is to do something like this:
let UserCanSeePII = current_principal_is_member_of('aadgroup=group1#domain.com');
let UserCanSeeFinance = current_principal_is_member_of('aadgroup=group2#domain.com');
let ResultWithPII = YourTable | where UserCanSeePII and (not UserCanSeeFinance) | where ... | extend ...;
let ResultWithFinance = YourTable | where UserCanSeeFinance and (not UserCanSeePII) | where ... | extend ...;
let ResultWithPIIandFinance = YourTable | where UserCanSeeFinance and UserCanSeePII | where ... | extend ...;
let ResultWithoutPIIandFinance = YourTable | where (not UserCanSeePII) and (not UserCanSeeFinance) | where ... | extend ...;
union ResultWithPII, ResultWithFinance, ResultWithPIIandFinance, ResultWithoutPIIandFinance
I have some json, for examle:
data = {
"name":"Bob","age":"20",
"name":"Jo","age":"21",
"name":"Jo","age":"22",
"name":"Nick","age":"23"
}
Next, I use crossfilter, create dimension and filter it:
let ndx = crossfilter(data);
let dim = ndx.dimension(d => d.name).filter(d !== "Jo");
//try to get filtered values
let filtered = dim.top(Infinity); // -> return 2 values where 'name'!='Jo'
//"name":"Bob","age":"20"
//"name":"Nick","age":"23"
let myGroup = dim.group(d => {
if(d === 'Jo') {
//Why we come here? This values must be filtered already
}
})
How can I filter my dimension and don't have these values on 'dim.group'?
Not sure what version you are using, but in the current version of Crossfilter, when a new group is created all records are first added to the group and then filtered records are removed. So the group accessor will be run at least once for all records.
Why do we do this? Because for certain types of grouping logic, it is important for the group to "see" a full picture of all records that are in scope.
It is possible that the group accessor is run over all records (even filtered ones) anyway in order to build the group index, but I don't remember.
I'm trying to get the following query in bookshelf's, any ideas? (this query is working and returns the required result)
SELECT "restaurants".*, "meals".*, ( select count(*) from "public"."visitors" as "visitors" where "visitors"."meal_id" = "meals"."id") as "visitorsMealsCount" FROM "public"."restaurants" as "restaurants" inner join "public"."meals" as "meals" ON "meals"."restaurant_id" = "restaurants"."id" WHERE "restaurants"."id" = '123'
Another question, after I'm using belongsTo and hasMany (for example) i expected the return object will be similar to that
restaurants (obj)
meals (obj)
visitorsMealsCount (attribute)
to me your SQL statement doesn't match the concept of an ORM tool like Bookshelf - it neither returns restaurants nor meals, but a grand mixture.
As to your second question: How about using new Restaurant({id: '123'}).fetch({ withRelated: ['meals']}) ?
visitorsMealsCount: In the documentation I see
adminAccounts: function() {
return this.belongsToMany(Account).query({where: {access: 'admin'}});
},
so maybe
visitorsMealsCount: function() {
return this.belongsToMany(Visitor).count();
}
works? (I haven't tried it.)
I have 2 List one stores the name of filterable columns(of type DropDown) and another store the values to load in those filterable columns.
List<string> filterableFields = new List<string>() { "A_B", "C_D", "E_F" };
List<string> AB, CD , EF;
Now at the run time I get the data from web service and I have written a function to to extract values for these filterable fields and store the values to 2nd List.
private void prepareListForFilterableColumns(XDocument records)
{
foreach (var currentField in filterableFields)
{
var values = (from xml in records.Descendants(z + "row")
let val = (string)xml.Attribute("ows_" + currentField.Replace("_", "_x0020_"))
where val != ""
orderby val
select val
).Distinct();
switch (currentField)
{
case "A_B": AB = values.ToList(); break;
case "C_D": CD = values.ToList(); break;
}
}
}
Now I was thinking that instead of hard coding the assignment in swtich case block, If I could just use the first List name "A_B" and replace "_" from it to point to my 2nd List and assign values.ToList() to it.
I understand that c# is a static language, So not sure if we can achieve this, but IF I can it will make my function generic.
Thanks a lot in advance for time and help.
Vishal
You could use a dictionary of lists of strings instead of 3 lists to store the values.
Dictionary<string, List<string>> val lists = new Dictionary<string,List<string>>();
And make the keys of the dictionary equal to the filterables: "AB", "CD",..
then, instead of AB you would use valLists["AB"] and could then reference reach list based on a string key.
The other option would be to use reflection but that would be slower and unnecessarily a bit more complicated.
I have a basic search control which lists companies from a CRM depending on predefined search/filtering criteria supplied by dropdowns. The default selection is "ALL" for each DropDown, otherwise the user chooses a specific item(s). I'd like to be able to construct a Linq query dynamically based on the selections. Out of the 5 selectors they supply values that I can match against the Company table, but two of the selectors (if either or both are chosen) would require a join or joins, else no action should be taken again the base result set. I hope this makes sense.
I'm not sure how to do this effectively. Here is my code:
private void Search()
{
EnergyPubsCRMDataContext dc = new EnergyPubsCRMDataContext();
var results = (from c in dc.Companies
select c);
//only create the join if the selected index > 0
if (ddlIndustry.SelectedIndex > 0)
{
//A company can be in 1 or more industries, thus here I want to join
//with the CompanyIndustry table and have a WHERE clause to match on the ddlIndustry.SelectedValue
}
//only create the join if the selected index > 0
if (ddlServices.SelectedIndex > 0)
{
//A company can offer 1 or more services. Here I want to join to the CompanyService table
//on the CompanyID and have a WHERE clause to match the ddlServices.SelectedValue
}
//These work OK to shape the overal query further (they don't need joins)
if (ddlCountry.SelectedIndex > 0)
results = results.Where(c => c.CountryID == Convert.ToInt32(ddlCountry.SelectedValue));
if (ddlStateRegion.SelectedIndex > 0)
results = results.Where(c => c.StateRegionID == Convert.ToInt32(ddlStateRegion.SelectedValue));
if (ddlAccountManagers.SelectedIndex > 0)
{
Guid g = new Guid(ddlAccountManagers.SelectedValue);
results = results.Where(c => c.UserId == g);
}
results = results.OrderBy(c => c.CompanyName);
//Bind to Grid....
}
if (ddlIndustry.SelectedIndex > 0)
{
//A company can be in 1 or more industries, thus here I want to join
//with the CompanyIndustry table and have a WHERE clause to match on the ddlIndustry.SelectedValue
results = results.Where(c => c.CompanyIndustry.IndustryID == ddlIndustry.SelectedValue);
}
Assuming you have correct foreign keys in your database/DBML.
This will generate the join implicitly.
I had very similar issue and no foreign keys I could leverage.
My solution would translate to something like this:
results = results
.Join(dc.CompanyIndustry, c => c.CompanyID, ci => ci.CompanyID, (c, ci) => new { c, ci.IndustryID })
.Where (a => a.IndustryID == ddlIndustry.SelectedValue)
.Select(a => a.c);
Basically:
1) first we create a join, with a projection that gives us IndustryID (join)
2) we filter based on IndustryID (where)
3) we return original anonymous type, so that we can modify original query (select)