Just recently i found out that the various implementations of VNC on Windows, Linux and other systems have an ID based method for negotiating the protocol method of the rfb-protocol to use for the communication. Some of the protocols have more encryption than others, some will only encrypt the logon credentials and some wont use any encryption at all.
Where can i find a (mostly complete) listing of the existing or standardized protocol IDs and their names? If a link to advanced information (e.g. specs) is attached to each of these entries i would appreciate this.
The RFB protocol registries are maintained by IANA.
Current Security Types:
0 Invalid [RFC6143]
1 None [RFC6143]
2 VNC Authentication [RFC6143]
3-15 RealVNC historic assignment
16 Tight historic assignment
17 Ultra historic assignment
18 TLS historic assignment
19 VeNCrypt historic assignment
20 GTK-VNC SASL historic assignment
21 MD5 hash authentication historic assignment
22 Colin Dean xvp historic assignment
23 Secure Tunnel [RFC7869]
24 Integrated SSH [RFC7869]
25-29 Unassigned
30-35 Apple Inc. [Michael_Stein]
36-127 Unassigned
128-255 RealVNC historic assignment
Current Encoding Types:
0 Raw [RFC6143]
1 CopyRect [RFC6143]
2 RRE [RFC6143]
5 Hextile [RFC6143]
9-10 Possibly used in UltraVNC historic assignment
16 ZRLE [RFC6143]
-239 Cursor pseudo-encoding [RFC6143]
-223 DesktopSize pseudo-encoding [RFC6143]
4 CoRRE historic assignment
6 zlib historic assignment
7 tight historic assignment
8 zlibhex historic assignment
15 TRLE [RFC6143]
17 Hitachi ZYWRLE historic assignment
20 H.264 [Tristan_Richardson]
21 JPEG historic assignment [Tristan_Richardson]
22 JRLE historic assignment [Tristan_Richardson]
1000-1002 Apple Inc. [Michael_Stein]
1011 Apple Inc. [Michael_Stein]
1024 to 1099 RealVNC historic assignment [Tristan_Richardson]
1100-1105 Apple Inc. [Michael_Stein]
-1 to -222 tight options historic assignment
-224 to -238 tight options historic assignment
-240 to -256 tight options historic assignment
-257 to -272 Anthony Liguori historic assignment
-273 to -304 VMWare historic assignment
-305 gii historic assignment
-306 popa historic assignment
-307 Peter Astrand DesktopName historic assignment
-308 Pierre Ossman ExtendedDesktopSize historic assignment
-309 Colin Dean xvp historic assignment
-310 OLIVE Call Control historic assignment
-311 ClientRedirect [Brian_Hinz]
-312 Fence TigerVNC
-313 ContinuousUpdates TigerVNC
-314 CursorWithAlpha [Tristan_Richardson]
-412 to -512 TurboVNC fine-grained quality level historic assignment
-523 to -528 Car Connectivity [Jörg_Brakensiek] [Terminal Mode v1.0]
-763 to -768 TurboVNC subsampling level historic assignment
0x48323634 VA H.264 [David_Verbeiren]
0x574d5600 to 0x574d56ff VMWare historic assignment
0xc0a1e5ce to 0xc0a1e5cf Possibly used in UltraVNC historic assignment
0xfffe0000 to 0xfffe00ff Used in LibVNCServer LibVNCServer
0xffff0000 to 0xffff8003 Possibly used in UltraVNC historic assignment
Related
Note: For convenience, PowerShell is used to demonstrate the behavior, but the question is about the behavior of the System.DateTimeOffset .NET type.
Note: As Matt Johnson points out, the behavior in question only happens on Unix-like platforms (macOS, Linux, using .NET Core).
Seemingly, when System.DateTimeOffset converts a local date, the resulting time-zone offset (offset from UTC) differs for dates before 19 Nov 1883.
Consider the following PowerShell command:
([datetime] '1883-11-18'), ([datetime] '1883-11-19') |
ForEach-Object { ([datetimeoffset] $_).Offset.ToString() }
which, when invoked in the Eastern Time Zone yields:
-04:57:00
-05:00:00
(DST (daylight-saving time) didn't come into play until 1918.)
I suspect that this is related to the following, excerpted from Wikipedia, emphasis added:
[...] had brought the US railway companies to an agreement which led to standard railway time being introduced at noon on 18 November 1883 across the nation.
Can anyone confirm this and provide more information?
Why, specifically, did the offset change by 3 minutes?
Searching for 1883 in the entire .NET documentation yields just the following, passing reference:
[...] the U.S. Eastern Standard Time zone from 1883 to 1917, before the introduction of daylight saving time in the United States
When running on Unix-like platforms (Linux, OSX, etc.), the time zone data in the operating system originates from the IANA time zone database. Early dates in this data set are referenced by their Local Mean Time (LMT), which is calculated from the latitude and longitude of the reference city. In this case, US Eastern Time being represented by America/New_York has an LMT entry that aligns with your reported findings.
From the tzdb:
# Zone NAME GMTOFF RULES FORMAT [UNTIL]
Zone America/New_York -4:56:02 - LMT 1883 Nov 18 12:03:58
-5:00 US E%sT 1920
-5:00 NYC E%sT 1942
-5:00 US E%sT 1946
-5:00 NYC E%sT 1967
-5:00 US E%sT
Note that seconds in the offset are truncated on import due to the precision of offsets supported by DateTimeOffset.
You can read more about LMT (and lots more) in the theory file in the tz database.
On Windows, you would not see this behavior, because Windows doesn't have time zone data for the Eastern Standard Time time zone earlier than the 2006-2007 transition. Thus dates earlier than 1987 may be converted incorrectly. If that matters to you, you'll need another source of time zone information, such as the copy of tzdb included with Noda Time.
Also note that local time is implied with casting from DateTime to DateTimeOffset, but only when the input value's .Kind is DateTimeKind.Local or DateTimeKind.Unspecified. If instead it's DateTimeKind.Utc, then the resulting DateTimeOffset will have a zero offset.
I have a quanteda corpus of hundreds of documents. How do I remove specific sections - like the abstract and footnotes etc. Otherwise, I am faced with doing it manually. Thanks
As requested, here is a text example. It is from a regular journal article. It shows the Meta data, then the abstract, then keywords, then introduction, then author contact details, then body of article, then Note, then Disclosure statement, then Notes on contributors, then references. I would like to remove everything apart from the introduction and body of the article. I would also like to remove the author name and Journal title - which are repeated throughout
" Behavioral Sciences of Terrorism and Political Aggression
ISSN: 1943-4472 (Print) 1943-4480 (Online) Journal homepage: http://www.tandfonline.com/loi/rirt20
Sometimes they come back: responding to
American foreign fighter returnees and other
Elusive threats
Christopher J. Wright
To cite this article: Christopher J. Wright (2018): Sometimes they come back: responding to
American foreign fighter returnees and other Elusive threats, Behavioral Sciences of Terrorism and
Political Aggression, DOI: 10.1080/19434472.2018.1464493
To link to this article: https://doi.org/10.1080/19434472.2018.1464493
Published online: 23 Apr 2018.
Submit your article to this journal
Article views: 57
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=rirt20
"
"BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION, 2018
https://doi.org/10.1080/19434472.2018.1464493
Sometimes they come back: responding to American foreign
fighter returnees and other Elusive threats
Christopher J. Wright
Department of Criminal Justice, Austin Peay State University, Clarksville, TN, USA
ABSTRACT ARTICLE HISTORY
Much has been made of the threat of battle hardened jihadis from Received 8 January 2018
Islamist insurgencies, especially Syria. But do Americans who Accepted 10 April 2018
return home after gaining experience fighting abroad pose a
KEYWORDS
greater risk than homegrown jihadi militants with no such Terrorism; foreign fighters;
experience? Using updated data covering 1990–2017, this study domestic terrorism;
shows that the presence of a returnee decreases the likelihood homegrown terrorism;
that an executed plot will cause mass casualties. Plots carried out lone-wolf; homeland security
Introduction: being afraid. Being a little afraid
How great of a threat do would-be jihadis pose to their home country? And do those who
return home after gaining experience fighting abroad in Islamist insurgencies or attending
terror training camps pose a greater risk than other jihadi militants? The fear, as first outlined
by Hegghammer (2013), is two-fold. First, individuals that have gone abroad to fight might
CONTACT Christopher J. Wright wrightc#apsu.edu Department of Criminal Justice, Austin Peay State University,
Clarksville, TN 37043, USA
© 2018 Society for Terrorism Research
"
"2 C. J. WRIGHT
Many of the earliest studies on Western foreign fighters suggested that those who
returned were in fact more deadly than those with no experience fighting in Islamist insur-
gencies. Hegghammer’s (2013) analysis suggested that these foreign fighter returnees
were a greater danger than when they left. Likewise, Byman (2015), Nilson (2015),
Kenney (2015), and Vidno (2011) came to similar conclusions while offering key insights
into the various mechanisms linking foreign fighting with successful plot execution and
greater mass casualties.
Other studies came to either mixed conclusions or directly contradicted the earlier find-
ings. Adding several years of data to Hegghammer’s (2013) earlier study, Hegghammer
"
" BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION 3
for them to form the types of large, local networks that would be necessary to carry out a
large-scale attack without attracting the attention of security services’ (p. 92).
"
Note
1. Charges were brought against Noor Zahi Salman, the widow of the Omar Mateen who carried
out the June, 2016 attack against the Pulse Nightclub in Orlando, Florida (US Department of
Justice., 2017a, January 17). However, in March of 2018 a jury acquitted her of the charges that
she had foreknowledge of the attack.
Disclosure statement
No potential conflict of interest was reported by the authors.
Notes on contributors
Christopher J. Wright, Ph.D., is an Assistant Professor at Austin Peay State University where he
teaches in the Homeland Security Concentration.
ORCID
Christopher J. Wright http://orcid.org/0000-0003-0043-6616
References
Byman, D. (2015). The homecomings: What happens when Arab foreign fighters in Iraq and Syria
return? Studies in Conflict & Terrorism, 38(8), 581–602.
Byman, D. (2016). The Jihadist returnee threat: Just how dangerous? Political Science Quarterly, 131(1),
69–99.
Byman, D., & Shapiro, J. (2014). Be afraid. Be a little afraid: The threat of terrorism from Western foreign
fighters in Syria and Iraq. Foreign Policy at Brookings. Washington, DC: Brookings. Retrieved from
https://www.brookings.edu/wp-content/uploads/2016/06/Be-Afraid-web.pdf
The approach
The key here is to determine the regular markers that precede each section, and then to use them as tags in a call to corpus_segment(). It's the tags that will need tweaking, based on their degree of regularity across documents.
Based on what you supplied above, I pasted that into a plain text file that I named example.txt. This code extracted the Introduction and what I think is the body of the article, but for that I had to decide a tag that marked its ending. Below, I used "Disclosure Statement". So:
library("quanteda")
crp <- readtext::readtext("~/tmp/example.txt") %>%
corpus()
pat <- c("\nIntroduction?", "\nCONTACT", "©", "\nDisclosure statement")
crpextracted <- corpus_segment(crp, pattern = pat)
summary(crpextracted)
## Corpus consisting of 4 documents:
##
## Text Types Tokens Sentences pattern
## example.txt.1 62 74 5 Introduction:
## example.txt.2 18 21 2 CONTACT
## example.txt.3 156 253 11 ©
## example.txt.4 101 180 19 Disclosure statement
##
## Source: /Users/kbenoit/Dropbox (Personal)/GitHub/quanteda/quanteda/* on x86_64 by kbenoit
## Created: Fri Jul 6 19:51:01 2018
## Notes: corpus_segment.corpus(crp, pattern = pat)
When you examine the text in the "Introduction:" tagged segment, you can see that everything from that string until the next tag was extracted to that as a new document:
corpus_subset(crpextracted, pattern == "\nIntroduction:") %>%
texts() %>% cat()
## being afraid. Being a little afraid
##
## How great of a threat do would-be jihadis pose to their home country? And do those who
##
## return home after gaining experience fighting abroad in Islamist insurgencies or attending
##
## terror training camps pose a greater risk than other jihadi militants? The fear, as first outlined
##
## by Hegghammer (2013), is two-fold. First, individuals that have gone abroad to fight might
How to remove pdf junk
All pdf conversions produce unwanted junk such as running headers, footers, etc. Here's how to remove them. (Note: You will want to do this before the step above.) How to construct the toreplace pattern? You will need to understand something about regular expressions, and use some experimentation.
library("stringr")
toreplace <- '\\n*\" \" BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION,{0,1} \\d+\\n*'
texts(crp) <- str_replace_all(texts(crp), regex(toreplace), "")
cat(texts(crp))
To demonstrate this on a section from your example:
# demonstration
x <- '
" " BEHAVIORAL SCIENCES OF TERRORISM AND POLITICAL AGGRESSION 3
'
str_replace_all(x, regex(toreplace), "")
## [1] ""
I've been tasked with looking into some issues regarding errors with unit conversions (Meters to Feet for example). Is there a definitive source for standard conversion numbers? I would like the conversion factors to be the highest precision possible.
I know there are a number of sites out there that offer a plethora of conversions, but what I am looking for is a standard, either official or defacto to use as a baseline.
A table with 32 and 64 bit numbers would be perfect.
The unit definition file of GNU Units lists a large number of sources:
Most units data was drawn from
NIST Special Publication 811, Guide for the
Use of the International System of Units (SI).
Barry N. Taylor. 1995
CRC Handbook of Chemistry and Physics 70th edition
Oxford English Dictionary
Websters New Universal Unabridged Dictionary
Units of Measure by Stephen Dresner
A Dictionary of English Weights and Measures by Ronald Zupko
British Weights and Measures by Ronald Zupko
Realm of Measure by Isaac Asimov
United States standards of weights and measures, their
creation and creators by Arthur H. Frazier.
French weights and measures before the Revolution: a
dictionary of provincial and local units by Ronald Zupko
Weights and Measures: their ancient origins and their
development in Great Britain up to AD 1855 by FG Skinner
The World of Measurements by H. Arthur Klein
For Good Measure by William Johnstone
NTC's Encyclopedia of International Weights and Measures
by William Johnstone
Sizes by John Lord
Sizesaurus by Stephen Strauss
CODATA Recommended Values of Physical Constants available at
http://physics.nist.gov/cuu/Constants/index.html
How Many? A Dictionary of Units of Measurement. Available at
http://www.unc.edu/~rowlett/units/index.html
Numericana. http://www.numericana.com
UK history of measurement
http://www.ukmetrication.com/history.htm
NIST Handbook 44, Specifications, Tolerances, and
Other Technical Requirements for Weighing and Measuring
Devices. 2011
NIST Special Publication 447, Weights and Measures Standards
of the the United States: a brief history. Lewis V. Judson.
1963; rev. 1976
For many units, the file contains the official definition, so it will be as precise as it gets. The comments often explain where that definition came from.
Before you say that this has already been asked, know that I've already reviewed these:
Is there a standard for storing normalized phone numbers in a database? - This is from 2008, and says that, at the time, there was no such standard. I'm hoping that something changed in the last 13 years.
How to validate phone numbers using regex - I already have the parse; it's quite easy: If it's not a digit, skip it. This question is not about the parser, but the format in which I save/display it. I am not worried about how hard it is to parse, but whether it's in standard format.
Say I'm working on a program that has to deal with phone numbers, and I want to make sure that they're saved and displayed in a standard format, so other programs and humans can also understand them predictably & consistently.
For instance, I've seen the following all be valid representations for the same US phone number:
1234567
123-4567
123 4567
5551234567
(555) 1234567
555-1234567
555 123 4567
555-123-4567
(555)-123-4567
(555) 123-4567
(5) 123 4567
1-555-123-4567
(1) 555-123-4567
+1 555-123-4567
+1 555 123-4567
+1 (555) 123-4567
Ad nauseum…
And then different countries represent numbers in different ways:
55 1234 567 8901
55 12 3456 7890
55 123 456 7890
55 1234 567890
555 123 456
(55) 123 4567
5.555.123-45-67
Ad nauseum…
As you can see, the number of ways a user can see a valid phone number is nearly infinite (The Wikipedia page for Telephone numbers in the UK is 26 printer pages long). I want all the numbers in my database and on the screen to be in a universally-recognizable format. As far as I can tell, ISO and ANSI have no defined format. Is there any standard notation for phone numbers?
There's no ISO standard, but there are ITU standards. You want E.123 and E.164.
In summary, any phone number is represented by +CC MMMMMM... where CC is the country code, and is one to three digits, and MMMMMM... is the area code (where applicable) and subscriber number. The total number of digits may not be more than 15. The + means "your local international dialling prefix".
I'll give a few examples using my own land line number.
So, for example, if you are in Germany, the number +44 2087712924 would be dialled as 00442087712924, and in the US you would dial it as 011442087712924. The 44 means that it is a UK number, and 2087712924 is the local part.
In practice, the long string of MMMMM... is normally broken up into smaller parts to make it easier to read. How you do that is country-specific. The example I give would normally be written +44 20 8771 2924.
As well as the unambiguous E.123 representation above, which you can use from anywhere in the world that allows international dialling, each country also has its own local method of representing numbers, and some have several. The example number will sometimes be written as 020 8771 2924 or (020) 8771 2924. The leading 0 is, strictly speaking, not part of the area code (that's 20) but a signal to the exchange meaning "here comes a number that could go outside the local area". Very occasionally the area code will be ommitted and the number will be written 8771 2924. All these local representations are ambiguous, as they could represent valid numbers in more than one country, or even valid numbers in more than one part of the same country. This means that you should always store a number with its country code, and ideally store it in E.123 notation. In particular you should note that phone numbers ARE NOT NUMBERS. A number like 05 is the same as 5. A phone number 05 is not the same as 5, and storage systems will strip leading zeroes from numbers. Store phone numbers as CHAR or VARCHAR in your database.
Finally, some oddities. The example number will be written by some stupid people as 0208 771 2924. This is diallable, but if you strip off the leading 0208 assuming that it is an area code, then the remainder is not valid as a local number. And some countries with broken phone systems [glares at North America] have utterly bonkers systems where in some places you must dial all 10 digits for a local call, some where you must not, some where you must dial 1NNN NNN NNNN, some where you must not include the leading one, and so on. In all such cases, storing the number as +CC MMMMM... is correct. It is up to someone actually making a call (or their dialling software) to figure out how to translate that into a dialable sequence of digits for their particular location.
There is a lot of local countries standards. On one of my projects I had the same problem. Solved as:
In DB everything stored as numbers: 123456789
Depending on selected web page language, this number pre-formatted when page loads.
Examples:
France, Luxemburg format phone numbers like 12.34.56.78.90 or 12 34
56 78 90
Germany: 123 45 67 or 123-45-67
Great Britain: 020 1234 1234 or 1234 01234 12345
Italy, Netherlands: 12 1234567 or 020-1234567
CIS countries: 123 123-45-67
I would like to think that some of the software I'm writing today will be used in 30 years. But I am also aware that a lot of it is based upon the UNIX tradition of exposing time as the number of seconds since 1970.
#include <stdio.h>
#include <time.h>
#include <limits.h>
void print(time_t rt) {
struct tm * t = gmtime(&rt);
puts(asctime(t));
}
int main() {
print(0);
print(time(0));
print(LONG_MAX);
print(LONG_MAX+1);
}
Execution results in:
Thu Jan 1 00:00:00 1970
Sat Aug 30 18:37:08 2008
Tue Jan 19 03:14:07 2038
Fri Dec 13 20:45:52 1901
The functions ctime(), gmtime(), and localtime() all take as an argument a time value representing the time in seconds since the Epoch (00:00:00 UTC, January 1, 1970; see time(3) ).
I wonder if there is anything proactive to do in this area as a programmer, or are we to trust that all software systems (aka Operating Systems) will some how be magically upgraded in the future?
Update It would seem that indeed 64-bit systems are safe from this:
import java.util.*;
class TimeTest {
public static void main(String[] args) {
print(0);
print(System.currentTimeMillis());
print(Long.MAX_VALUE);
print(Long.MAX_VALUE + 1);
}
static void print(long l) {
System.out.println(new Date(l));
}
}
Wed Dec 31 16:00:00 PST 1969
Sat Aug 30 12:02:40 PDT 2008
Sat Aug 16 23:12:55 PST 292278994
Sun Dec 02 08:47:04 PST 292269055
But what about the year 292278994?
I have written portable replacement for time.h (currently just localtime(), gmtime(), mktime() and timegm()) which uses 64 bit time even on 32 bit machines. It is intended to be dropped into C projects as a replacement for time.h. It is being used in Perl and I intend to fix Ruby and Python's 2038 problems with it as well. This gives you a safe range of +/- 292 million years.
You can find the code at the y2038 project. Please feel free to post any questions to the issue tracker.
As to the "this isn't going to be a problem for another 29 years", peruse this list of standard answers to that. In short, stuff happens in the future and sometimes you need to know when. I also have a presentation on the problem, what is not a solution, and what is.
Oh, and don't forget that many time systems don't handle dates before 1970. Stuff happened before 1970, sometimes you need to know when.
You can always implement RFC 2550 and be safe forever ;-)
The known universe has a finite past and future. The current age of
the universe is estimated in [Zebu] as between 10 ** 10 and 2 * 10 **
10 years. The death of the universe is estimated in [Nigel] to occur
in 10 ** 11 - years and in [Drake] as occurring either in 10 ** 12
years for a closed universe (the big crunch) or 10 ** 14 years for an
open universe (the heat death of the universe).
Y10K compliant programs MAY choose to limit the range of dates they
support to those consistent with the expected life of the universe.
Y10K compliant systems MUST accept Y10K dates from 10 ** 12 years in
the past to 10 ** 20 years into the future. Y10K compliant systems
SHOULD accept dates for at least 10 ** 29 years in the past and
future.
Visual Studio moved to a 64 bit representation of time_t in Visual Studio 2005 (whilst still leaving _time32_t for backwards compatibility).
As long as you are careful to always write code in terms of time_t and don't assume anything about the size then as sysrqb points out the problem will be solved by your compiler.
Given my age, I think I should pay a lot into my pension and pay of all my depts, so someone else will have to fit the software!
Sorry, if you think about the “net present value” of any software you write today, it has no effect what the software does in 2038. A “return on investment” of more than a few years is uncommon for any software project, so you make a lot more money for your employer by getting the software shipped quicker, rather than thinking that far ahead.
The only common exception is software that has to predict future, 2038 is already a problem for mortgage quotation systems.
I work in embedded and I thought I would post our solution here. Our systems are on 32 bits, and what we sell right now has a warantee of 30 years which means that they will encounter the year 2038 bug. Upgrading in the future was not a solution.
To fix this, we set the kernel date 28 years earlier that the current date. It's not a random offset, 28 years is excatly the time it will take for the days of the week to match again. For instance I'm writing this on a thursday and the next time march 7 will be a thursday is in 28 years.
Furthermore, all the applications that interact with dates on our systems will take the system date (time_t) convert it to a custom time64_t and apply the 28 years offset to the right date.
We made a custom library to handle this. The code we're using is based off this: https://github.com/android/platform_bionic
Thus, with this solution you can buy yourself an extra 28 years easily.
I think that we should leave the bug in. Then about 2036 we can start selling consultancy for large sums of money to test everything. After all isn't that how we successfully managed the 1999-2000 rollover.
I'm only joking!
I was sat in a bank in London in 1999 and was quite amazed when I saw a consultant start Y2K testing the coffee machine. I think if we learnt anything from that fiasco, it was that the vast majority of software will just work and most of the rest won't cause a melt down if it fails and can be fixed after the event if needed. As such, I wouldn't take any special precautions until much nearer the time, unless you are dealing with a very critical piece of software.
Keep good documentation, and include a description of your time dependencies. I don't think many people have thought about how hard this transition might be, for example HTTP cookies are going to break on that date.
What should we do to prepare for 2038?
Hide, because the apocalypse is coming.
But seriously, I hope that compilers (or the people who write them, to be precise) can handle this. They've got almost 30 years. I hope that's enough time.
At what point do we start preparing for Y10K? Have any hardware manufacturers / research labs looked into the easiest way to move to whatever new technology we'll have to have because of it?
By 2038, time libraries should all be using 64-bit integers, so this won't actually be that big of a deal (on software that isn't completely unmaintained).
COBOL programs might be fun though.
Operative word being "should".
If you need to ensure futureproofing then you can construct your own date/time class and use that but I'd only do that if you think that what you write will be used on legacy OS'