I am trying to understand the given protocol of recommenderlab library in R.
From the original document https://cran.r-project.org/web/packages/recommenderlab/vignettes/recommenderlab.pdf :
Testing is perfomed by withholding items (parameter given). Breese et
al. (1998) introduced the four experimental witholding protocols
called Given 2, Given 5, Given 10 and All-but-1. During testing, the
Given x protocol presents the algorithm with only x randomly chosen
items for the test user, and the algorithm is evaluated by how well it
is able to predict the withheld items
I do not understand the fact that this is only applied to the testing users. Why can't we simply apply this to all users, and we need to split between training and testing users AND use given protocol?
Since we give only few items for the algorithm to understand patterns and produce predictions, and then test/measure how good are those predictions based on the rest of the items, why do we need to do this for the test users only, and we can't do it for all? How do the training users serve? When the algorithm is given let's say 10 items to understand the target user's patterns, doesn't it use the whole dataset (both training and testing) to compute the user neighborhood for example (UBCF) in order to make the predictions that will be later evaluated on using the withheld items? And if not, meaning that during this process it only looks at the training users (say 80% for instance) why wouldn't it look to the testing users as well? What is the problem of having a another test user in the neighborhood, and there needs to be only training users? That part I don't get.. Or my assumptions are wrong?
To conclude: Why do we both need the given protocol, as well as the split between training and testing?
I hope my question makes sense, I am really curious to find the solution. Thank you in advance :)
I was wondering wether SCORM can store multiple scores of a SCO. I have read many articles and same state it can't, and when I restart a SCO it erases the previous score, other articles sais the opposite. How is it? Maybe the 1.2 can't, and the 2004 version can?
Well...it's complicated. You can restart a SCO, but it wipes the tracking data and score and starts fresh.
From the SCORM 2004 4th ed docs:
4.2.8. Exit
The cmi.exit data model element indicates how or why the learner left the SCO [1].
This value is used to indicate the reason
that the SCO was last exited. The cmi.exit data model element is
involved with temporal aspects of the run-time execution of the SCO.
• If the cmi.exit is set to “suspend” then the SCOs current learner
attempt does not end. The SCOs Run-Time Environment data model
element values for the current learner session will be available to
the SCO if the SCO is relaunched in a subsequent learner session.
• If the cmi.exit is set to “normal”, “logout”,“time-out” or “” (empty
characterstring) then the SCOs learner attempt ends. The SCOs
Run-Time Environment data model element values of the current learner
session will NOT be available if the SCO is relaunched.
ADL Note: If an LMS invokes a Suspend All navigation request, then the value of
cmi.exit will be ignored. In these cases, the SCOs current learner
attempt does not end. The SCO’s data model element values shall be
persisted and available to the SCO if the SCO is relaunched.
But if you want to allow learners to have multiple scores for the same SCO, that isn't supported in the SCORM standard as far as I know. It doesn't mean that the LMS couldn't offer such a feature, though, to create "pseudo-learners" that map to the same person, letting you keep multiple attempts. Or, if your LMS has an API or supports webhooks, you could also create your own separate datastore that records scores, so that if the SCO gets wiped and restarted you can still keep the historical data on your own. You would need to check with your LMS vendor to see if either of those options would be supported.
You'll want to look into interactions. That gives you greater ability to journal or update the activities/questions/interactions of your content. That will also enable you to set the result, latency and other values per interaction.
SCORM 1.2 was mostly an optionally supported standard but it did support it. SCORM 2004 made it mandatory so its more likely you'll have the support.
There is a lot of wiki based info on this located here: https://github.com/cybercussion/SCOBot/wiki/SCORM-SCOBot-Documentation#set-interaction
Just remember the SCOBot Content API is a javascript library to interface the LMS Runtime API. It includes the roll-up of the white paper so it's easier to work with all facets of the SCORM Specification. It is obviously not a replacement for reading the specification.
I also recognize you may be asking about attempts here. That would be up to the LMS to record that. SCORM does not make any direct restrictions if the LMS does/does not do that or is required to.
Can somebody please refer me how do we separate out the models while separating out each service to run of their own? So basically a few of the models we have right now overlaps among services. I went through some of the procedures which ask to use canonical pattern, however I also got that we should not use canonical pattern.
One of the solution would be to keep all the models in a common place which we are doing right now. But its seems problematic for managing the services in the form of one repository per one service.
Duplicate models are also fine with me if I can find good logic for it.
Vertical slicing and domain decomposition leads to having each vertical slice have it's own well defined fields (not entities) that belong together (bounded context/business functions), defining a logical service boundary and decomposing the Service Boundary to Business Components and Autonomous Components (smallest unit of work).
Each component owns the data it modifies and is the only one in the system that can change the state of that data, you can have many copies of the data for readers but only one logical writer.
There for it makes more sense not to share these data models as they are internal to the component responsible for the data model's state (and is encapsulated).
Data readers can use view models and these are dictated more by the consumer then the producer, the view/read models hold the "real"/"current" state of the transactional data (which is private to the data modifier), read data can be updated by the data processor after any logical state change.
So it makes more sense to publish view/read models for read only consumers...
Check out Udi Dahan's video
Does that make sense?
I'm beginning the process of instrumenting a web application, and using StatsD to gather as many relevant metrics as possible. For instance, here are a few examples of the high-level metric names I'm currently using:
http.responseTime
http.status.4xx
http.status.5xx
view.renderTime
oauth.begin.facebook
oauth.complete.facebook
oauth.time.facebook
users.active
...and there are many, many more. What I'm grappling with right now is establishing a consistent hierarchy and set of naming conventions for the various metrics, so that the current ones make sense and that there are logical buckets within which to add future metrics.
My question is two fold:
What relevant metrics are you gathering that you have found indespensible?
What naming structure are you using to categorize metrics?
This is a question that has no definitive answer but here's how we do it at Datadog (we are a hosted monitoring service so we tend to obsess over these things).
1. Which metrics are indispensable? It depends on the beholder. But at a high-level, for each team, any metric that is as close to their goals as possible (which may not be the easiest to gather).
System metrics (e.g. system load, memory etc.) are trivial to gather but seldom actionable because they are too hard to reliably connect them to a probable cause.
On the other hand number of completed product tours matter to anyone tasked with making sure new users are happy from the first minute they use the product. StatsD makes this kind of stuff trivially easy to collect.
We have also found that the core set of key metrics for any teamchanges as the product evolves so there is a continuous editorial process.
Which in turn means that anyone in the company needs to be able to pick and choose which metrics matter to them. No permissions asked, no friction to get to the data.
2. Naming structure The highest level of hierarchy is the product line or the process. Our web frontend is internally called dogweb so all the metrics from that component are prefixed with dogweb.. The next level of hierarchy is the sub-component, e.g. dogweb.db., dogweb.http., etc.
The last level of hierarchy is the thing being measured (e.g. renderTime or responseTime).
The unresolved issue in graphite is the encoding of metric metadata in the metric name (and selection using *, e.g. dogweb.http.browser.*.renderTime) It's clever but can get in the way.
We ended up implementing explicit metadata in our data model, but this is not in statsd/graphite so I will leave the details out. If you want to know more, contact me directly.
I apologize as I don't know whether this is more of a math question that belongs on mathoverflow or if it's a computer science question that belongs here.
That said, I believe I understand the fundamental difference between data, information, and knowledge. My understanding is that information carries both data and meaning. One thing that I'm not clear on is whether information is data. Is information considered a special kind of data, or is it something completely different?
The words data,information and knowlege are value-based concepts used to categorize, in a subjective fashion, the general "conciseness" and "usefulness" of a particular information set.
These words have no precise meaning because they are relative to the underlying purpose and methodology of information processing; In the field of information theory these have no meaning at all, because all three are the same thing: a collection of "information" (in the Information-theoric sense).
Yet they are useful, in context, to summarize the general nature of an information set as loosely explained below.
Information is obtained (or sometimes induced) from data, but it can be richer, as well a cleaner (whereby some values have been corrected) and "simpler" (whereby some irrelevant data has been removed). So in the set theory sense, Information is not a subset of Data, but a separate set [which typically intersects, somewhat, with the data but also can have elements of its own].
Knowledge (sometimes called insight) is yet another level up, it is based on information and too is not a [set theory] subset of information. Indeed Knowledge typically doesn't have direct reference to information elements, but rather tells a "meta story" about the information / data.
The unfounded idea that along the Data -> Information -> Knowledge chain, the higher levels are subsets of the lower ones, probably stems from the fact that there is [typically] a reduction of the volume of [IT sense] information. But qualitatively this info is different, hence no real [set theory] subset relationship.
Example:
Raw stock exchange data from Wall Street is ... Data
A "sea of data"! Someone has a hard time finding what he/she needs, directly, from this data. This data may need to be normalized. For example the price info may sometimes be expressed in a text string with 1/32th of a dollar precision, in other cases prices may come as a true binary integer with 1/8 of a dollar precision. Also the field which indicate, say, the buyer ID, or seller ID may include typos, and hence point to the wrong seller/buyer. etc.
A spreadsheet made from the above is ... Information
Various processes were applied to the data:
-cleaning / correcting various values
-cross referencing (for example looking up associated codes such as adding a column to display the actual name of the individual/company next to the Buyer ID column)
-merging when duplicate records pertaining to the same event (but say from different sources) are used to corroborate each other, but are also combined in one single record.
-aggregating: for example making the sum of all transaction value for a given stock (rather than showing all the individual transactions.
All this (and then some) turned the data into Information, i.e. a body of [IT sense] Information that is easily usable, where one can quickly find some "data", such as say the Opening and Closing rate for the IBM stock on June 8th 2009.
Note that while being more convenient to use, in part more exact/precise, and also boiled down, there is not real [IT sense] information in there which couldn't be located or computed from the original by relatively simple (if only painstaking) processes.
An financial analyst's report may contain ... knowledge
For example if the report indicate [bogus example] that whenever the price of Oil goes past a certain threshold, the value of gold start declining, but then quickly spikes again, around the time the price of coffee and tea stabilize. This particular insight constitute knowledge. This knowledge may have been hidden in the data alone, all along, but only became apparent when one applied some fancy statistically analysis, and/or required the help of a human expert to find or confirm such patterns.
By the way, in the Information Theory sense of the word Information, "data", "information" and "knowlegde" all contain [IT sense] information.
One could possibly get on the slippery slope of stating that "As we go up the chain the entropy decreases", but that is only loosely true because
entropy decrease is not directly or systematically tied to "usefulness for human"
(a typical example is that a zipped text file has less entropy yet is no fun reading)
there is effectively a loss of information (in addition to entropy loss)
(for example when data is aggregate the [IT sense] information about individual records get lost)
there is, particular in the case of Information -> Knowlege, a change in level of abstration
A final point (if I haven't confused everybody yet...) is the idea that the data->info->knowledge chain is effectively relative to the intended use/purpose of the [IT-sense] Information.
ewernli in a comment below provides the example of the spell checker, i.e. when the focus is on English orthography, the most insightful paper from a Wallstreet genius is merely a string of words, effectively "raw data", some of it in need of improvement (along the orthography purpose chain.
Similarly, a linguist using thousands of newspaper articles which typically (we can hope...) contain at least some insight/knowledge (in the general sense), may just consider these articles raw data, which will help him/her create automatically French-German lexicon (this will be information), and as he works on the project, he may discover a systematic semantic shift in the use of common words betwen the two languages, and hence gather insight into the distinct cultures.
Define information and data first, very carefully.
What is information and what is data is very dependent on context. An extreme example is a picture of you at a party which you email. For you it's information, but for the the ISP it's just data to be passed on.
Sometimes just adding the right context changes data to information.
So, to answer you question: No, information is not a subset of data. It could be at least the following.
A superset, when you add context
A subset, needle-in-a-haystack issue
A function of the data, e.g. in a digest
There are probably more situations.
This is how I see it...
Data is dirty and raw. You'll probably have too much of it.
... Jason ... 27 ... Denton ...
Information is the data you need, organised and meaningful.
Jason.age=27
Jason.city=Denton
Knowledge is why there are wikis, blogs: to keep track of insights and experiences. Note that these are human (and community) attributes. Except for maybe a weird science project, no computer is on Facebook telling people what it believes in.
information is an enhancement of data:
data is inert
information is actionable
note that information without data is merely an opinion ;-)
Information could be data if you had some way of representing the additional content that makes it information. A program that tries to 'understand' written text might transform the input text into a format that allows for more complex processing of the meaning of that text. This transformed format is a kind of data that represents information, when understood in the context of the overall processing system. From outside the system it appears as data, whereas inside the system it is the information that is being understood.