I'm writing a script that uploads a file to a cgi script that expects a multipart request, such as a form on a HTML page. The boundary is a unique token that annotates the file contents in the request body. Here's an example body:
--BOUNDARY
Content-Disposition: form-data; name="paramname"; filename="foo.txt"
Content-Type: text/plain
... file contents here ...
--BOUNDARY--
The boundary cannot be present in the file contents, for obvious reasons.
What should I do in order to create an unique boundary? Should I generate a random string, check to see if it is in the file contents, and if it is, generate a new, rinse and repeat, until I have a unique string? Or would a "pretty random token" (say, combination of timestamp, process id, etc) be enough?
If you use something random enough like a GUID there shouldn't be any need to hunt through the payload to check for an alias of the boundary. Something like:-
----=NextPart_3676416B-9AD6-440C-B3C8-FC66DDC7DB45
Header:....
Payload
----=NextPart_3676416B-9AD6-440C-B3C8-FC66DDC7DB45--
For Java guys :
protected String generateBoundary() {
StringBuilder buffer = new StringBuilder();
Random rand = new Random();
int count = rand.nextInt(11) + 30; // a random size from 30 to 40
for (int i = 0; i < count; i++) {
buffer.append(MULTIPART_CHARS[rand.nextInt(MULTIPART_CHARS.length)]);
}
return buffer.toString();
}
private final static char[] MULTIPART_CHARS =
"-_1234567890abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
.toCharArray();
Reference url : http://hc.apache.org/httpcomponents-client-ga/httpmime/xref/org/apache/http/entity/mime/MultipartEntity.html
And for the Swift people (to balance the Java):
func createBoundaryString() -> String {
var str = ""
let length = arc4random_uniform(11) + 30
let charSet = [Character]("-_1234567890abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
for _ in 0..<length {
str.append(charSet[Int(arc4random_uniform(UInt32(charSet.count)))])
}
return str
}
If you are feeling paranoid, you can generate a random boundary and search for it in the string to be sent, append random char (or re-create new) on find, repeat. But my experience is any arbitrary non-dictionary string of 10 or so characters is about impossible to occur, so picking something like ---BOUNDARY---BOUNDARY---BOUNDARY--- is perfectly sufficient.
Related
I recently saw code for reading entire contents of an InputStream into a String in Kotlin, such as:
// input is of type InputStream
val baos = ByteArrayOutputStream()
input.use { it.copyTo(baos) }
val inputAsString = baos.toString()
And also:
val reader = BufferedReader(InputStreamReader(input))
try {
val results = StringBuilder()
while (true) {
val line = reader.readLine()
if (line == null) break
results.append(line)
}
val inputAsString = results.toString()
} finally {
reader.close()
}
And even this that looks smoother since it auto-closes the InputStream:
val inputString = BufferedReader(InputStreamReader(input)).useLines { lines ->
val results = StringBuilder()
lines.forEach { results.append(it) }
results.toString()
}
Or slight variation on that one:
val results = StringBuilder()
BufferedReader(InputStreamReader(input)).forEachLine { results.append(it) }
val resultsAsString = results.toString()
Then this functional fold thingy:
val inputString = input.bufferedReader().useLines { lines ->
lines.fold(StringBuilder()) { buff, line -> buff.append(line) }.toString()
}
Or a bad variation which doesn't close the InputStream:
val inputString = BufferedReader(InputStreamReader(input))
.lineSequence()
.fold(StringBuilder()) { buff, line -> buff.append(line) }
.toString()
But they are all clunky and I keep finding newer and different versions of the same... and some of them never even close the InputStream. What is a non-clunky (idiomatic) way to read the InputStream?
Note: this question is intentionally written and answered by the author (Self-Answered Questions), so that the idiomatic answers to commonly asked Kotlin topics are present in SO.
Kotlin has a specific extension just for this purpose.
The simplest:
val inputAsString = input.bufferedReader().use { it.readText() } // defaults to UTF-8
And in this example, you could decide between bufferedReader() or just reader(). The call to the function Closeable.use() will automatically close the input at the end of the lambda's execution.
Further reading:
If you do this type of thing a lot, you could write this as an extension function:
fun InputStream.readTextAndClose(charset: Charset = Charsets.UTF_8): String {
return this.bufferedReader(charset).use { it.readText() }
}
Which you could then call easily as:
val inputAsString = input.readTextAndClose() // defaults to UTF-8
On a side note, all Kotlin extension functions that require knowing the charset already default to UTF-8, so if you require a different encoding you need to adjust the code above in calls to include encoding for reader(charset) or bufferedReader(charset).
Warning: You might see examples that are shorter:
val inputAsString = input.reader().readText()
But these do not close the stream. Make sure you check the API documentation for all of the IO functions you use to be sure which ones close and which do not. Usually, if they include the word use (such as useLines() or use()) they close the stream after. An exception is that File.readText() differs from Reader.readText() in that the former does not leave anything open and the latter does indeed require an explicit close.
See also: Kotlin IO related extension functions
【Method 1 | Manually Close Stream】
private fun getFileText(uri: Uri):String {
val inputStream = contentResolver.openInputStream(uri)!!
val bytes = inputStream.readBytes() //see below
val text = String(bytes, StandardCharsets.UTF_8) //specify charset
inputStream.close()
return text
}
inputStream.readBytes() requires manually close the stream: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/java.io.-input-stream/read-bytes.html
【Method 2 | Automatically Close Stream】
private fun getFileText(uri: Uri): String {
return contentResolver.openInputStream(uri)!!.bufferedReader().use {it.readText() }
}
You can specify the charset inside bufferedReader(), default is UTF-8:
https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/java.io.-input-stream/buffered-reader.html
bufferedReader() is an upgrade version of reader(), it is more versatile:
How exactly does bufferedReader() work in Kotlin?
use() can automatically close the stream when the block is done:
https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/use.html
An example that reads contents of an InputStream to a String
import java.io.File
import java.io.InputStream
import java.nio.charset.Charset
fun main(args: Array<String>) {
val file = File("input"+File.separator+"contents.txt")
var ins:InputStream = file.inputStream()
var content = ins.readBytes().toString(Charset.defaultCharset())
println(content)
}
For Reference - Kotlin Read File
Quick solution works well when converting InputStream to string.
val convertedInputStream = String(inputStream.readAllBytes(), StandardCharsets.UTF_8)
HTTPServletRequest req, has a method getParameterMap() but, the values return a String[] instead of String, for post data as
name=Marry&lastName=John&Age=20.
I see in the post data it's not an array, but getParameterMap() returns array for every key(name or lastName or Age). Any pointers on understanding this in a better way?
The code is available in Approach 2. Approach 1 works completely fine.
Approach 1:
Enumeration<String> parameterNames = req.getParameterNames();
while (parameterNames.hasMoreElements()) {
String key = (String) parameterNames.nextElement();
String val = req.getParameter(key);
System.out.println("A= <" + key + "> Value<" + val + ">");
}
Approach 2:
Map<String, Object> allMap = req.getParameterMap();
for (String key : allMap.keySet()) {
String[] strArr = (String[]) allMap.get(key);
for (String val : strArr) {
System.out.println("Str Array= " + val);
}
}
If you are expecting pre determined parameters then you can use getParameter(java.lang.String name) method.
Otherwise, approaches given above can be used, but with some differences, in HTTP-request someone can send one or more parameters with the same name.
For example:
name=John, name=Joe, name=Mia
Approach 1 can be used only if you expect client sends only one parameter value for a name, rest of them will be ignored. In this example you can only read "John"
Approach 2 can be used if you expect more than one values with same name. Values will be populated as an array as you showed in the code. Hence you will be able to read all values, i.e "John","Joe","Mia" in this example
Documentation
In the YSOD below, the stacktrace (and the source file line) contain the full path to the source file. Unfortunately, the full path to the source file name contains my user name, which is firstname.lastname.
I want to keep the YSOD, as well as the stack trace including the filename and line number (it's a demo and testing system), but the username should vanish from the sourcefile path. Seeing the file's path is also OK, but the path should be truncated at the solution root directory.
(without me having to copy-paste the solution every time to another path before publishing it...)
Is there any way to accomplish this ?
Note: Custom error pages aren't an option.
Path is embedded in .pdb files, which are produced by the compiler. The only way to change this is to build your project in some other location, preferably somewhere near the build server.
Never mind, I found it out myself.
Thanks to Anton Gogolev's statement that the path is in the pdb file, I realized it is possible.
One can do a binary search-and-replace on the pdb file, and replace the username with something else.
I quickly tried using this:
https://codereview.stackexchange.com/questions/3226/replace-sequence-of-strings-in-binary-file
and it worked (on 50% of the pdb files).
So mind the crap, that code-snippet in the link seems to be buggy.
But the concept seems to work.
I now use this code:
public static void SizeUnsafeReplaceTextInFile(string strPath, string strTextToSearch, string strTextToReplace)
{
byte[] baBuffer = System.IO.File.ReadAllBytes(strPath);
List<int> lsReplacePositions = new List<int>();
System.Text.Encoding enc = System.Text.Encoding.UTF8;
byte[] baSearchBytes = enc.GetBytes(strTextToSearch);
byte[] baReplaceBytes = enc.GetBytes(strTextToReplace);
var matches = SearchBytePattern(baSearchBytes, baBuffer, ref lsReplacePositions);
if (matches != 0)
{
foreach (var iReplacePosition in lsReplacePositions)
{
for (int i = 0; i < baReplaceBytes.Length; ++i)
{
baBuffer[iReplacePosition + i] = baReplaceBytes[i];
} // Next i
} // Next iReplacePosition
} // End if (matches != 0)
System.IO.File.WriteAllBytes(strPath, baBuffer);
Array.Clear(baBuffer, 0, baBuffer.Length);
Array.Clear(baSearchBytes, 0, baSearchBytes.Length);
Array.Clear(baReplaceBytes, 0, baReplaceBytes.Length);
baBuffer = null;
baSearchBytes = null;
baReplaceBytes = null;
} // End Sub ReplaceTextInFile
Replace firstname.lastname with something that has equally many characters, for example "Poltergeist".
Now I only need to figure out how to run the binary search and replace as a post-build action.
I want to combine two guid values and generate a 32 bit alphanumberic value(It can be done by using hashing).
Not Pretty, but it works..
private static Guid MungeTwoGuids(Guid guid1, Guid guid2)
{
const int BYTECOUNT = 16;
byte[] destByte = new byte[BYTECOUNT];
byte[] guid1Byte = guid1.ToByteArray();
byte[] guid2Byte = guid2.ToByteArray();
for (int i = 0; i < BYTECOUNT; i++)
{
destByte[i] = (byte) (guid1Byte[i] ^ guid2Byte[i]);
}
return new Guid(destByte);
}
and yes, I can deal with the non-unique-guarantee in my case
What about splitting the Guids into 2 chunks of 8 bytes each, convert them to ulong (8 bytes), XOR combine them and then concat the 2 results.
public static Guid Combine(this Guid x, Guid y)
{
byte[] a = x.ToByteArray();
byte[] b = y.ToByteArray();
return new Guid(BitConverter.GetBytes(BitConverter.ToUInt64(a, 0) ^ BitConverter.ToUInt64(b, 8))
.Concat(BitConverter.GetBytes(BitConverter.ToUInt64(a, 8) ^ BitConverter.ToUInt64(b, 0))).ToArray());
}
You can't convert 2 128-bit GUIDs into a 16-bit or 32-bit value and maintain uniqueness. For your stated application (use value in URL) this doesn't seem to make sense, as a given value in the URL could map to any number of GUID combinations. Have you considered this?
The best approach would be to use an URL-shortening lookup where you generate a unique ID and map it to the GUIDs if needed - similarly to bit.ly or tinyurl.com.
var a = Guid.NewGuid();
var b = Guid.NewGuid();
var hashOfXor = Xor(a, b).GetHashCode();
public static Guid Xor(Guid a, Guid b)
{
unsafe
{
Int64* ap = (Int64*) &a;
Int64* bp = (Int64*) &b;
ap[0] ^= bp[0];
ap[1] ^= bp[1];
return *(Guid*) ap;
}
}
I actually did have a need to merge two Guids together to create a third Guid.
Where the third Guid (not necessarily unique) would be the same regardless of the order the two original Guids were supplied.
So I came up with this:
public static Guid Merge(Guid guidA, Guid guidB)
{
var aba = guidA.ToByteArray();
var bba = guidB.ToByteArray();
var cba = new byte[aba.Length];
for (var ix = 0; ix < cba.Length; ix++)
{
cba[ix] = (byte)(aba[ix] ^ bba[ix]);
}
return new Guid(cba);
}
Assuming you want to generate a 32 byte value you can just concatenate the GUIDs since they are 16 byte each. If you really need a 32 bit value the only solution I see is generating your own 32 bit values and storing the related GUIDs in a database so you can retrieve them later.
In .NET Core 3 we can use Sse2/Span<T> to speed things up, and avoid all allocations. Essentially this code treats a Guid as 2 consecutive Int64 values, and performs the xor on them. SSE2 performs the xor in a single processor instruction (SIMD).
public static Guid Xor(this Guid a, Guid b)
{
if (Sse2.IsSupported)
{
var result = Sse2.Xor(Unsafe.As<Guid, Vector128<long>>(ref a), Unsafe.As<Guid, Vector128<long>>(ref b));
return Unsafe.As<Vector128<long>, Guid>(ref result);
}
var spanA = MemoryMarshal.CreateSpan(ref Unsafe.As<Guid, long>(ref a), 2);
var spanB = MemoryMarshal.CreateSpan(ref Unsafe.As<Guid, long>(ref b), 2);
spanB[0] ^= spanA[0];
spanB[1] ^= spanA[1];
return b;
}
Depends on the platform and details of what you are trying to do.
In .NET/C# you could jus take avery simple approach:
var result = g1.GetHashCode() ^ g2.GetHashCode();
I would use an UUID5 (name-based) to combine two GUIDs, see https://stackoverflow.com/a/5657517/7556646
Guid g1 = new Guid("6164742b-e171-471b-ad6f-f98a78c5557e");
Guid g2 = new Guid("acbc41aa-971c-422a-bd42-bbcefa32ffb4");
Guid g12 = Create(IsoOidNamespace, g1.ToString() + g2.ToString(), 5)
In this example g12 would be: e1ccaee5-ea5e-55c6-89a5-fac02043326e.
There's no native support in the .NET Framework for creating these, but the code is posted on GitHub that implements the algorithm.
See as well the following .NET Fiddle, https://dotnetfiddle.net/VgHLtz
Why not try a simple operator i.e. AND, OR, XOR etc. To combine the two. XOR would be your best bet hear I would imagine as it has the nice property of when xoring the result with either of the two inputs you will get the other.
Edit: having just looked at this solution, there is a problem with it. The values would have to be normalised. Take a look at Vinay's Answer for a better solution.
Here's a one-liner for you:
g1.ToByteArray().Concat(g2.ToByteArray()).GetHashCode()
public static string Merge(Guid one, Guid two)
{
return new List<Guid>() { one, two }
.OrderBy(x => x.GetHashCode())
.Select(y => y.ToString().ToLowerInvariant())
.Aggregate((a, b) => ${a.ToLowerInvariant()}_{b.ToLowerInvariant()}");
}
So in my situation i needed to maintain order in order to make sure that the 2 Guids could be merged regardless of order. Therefore they have to be ordered. That was step one. Then, it's simply selecting the guids to string and for consitency (super important), I used string.ToLowerInvariant(). Then concatenated them using the .Aggregate function.
I am having some sort of problem with encoding in my ASP.NET HTTPHandler, which uploads a file. The file content is passed in a hidden form variable from a ColdFusion web page which is using something called "ToBase64".
In ColdFusion, the code used to place the file content into a form is as follows:
<cffile action="readBinary" file="#FileName#" variable="objBinaryData">
<cfset b64file = #toBase64(objBinaryData)#>
<form name="sendToHandler"
action="http://myserver/mysite/UploadHandler.ashx" method="post">
<cfoutput>
<input type="hidden" name="objBinaryData" value="#b64file#" />
When my UploadHandler.ashx is posted, I am getting a string out of the form as follows:
string fileContent = context.Request.Form["objBinaryData"];
Next, I am converting the string to a byte array as follows:
byte[] binData = StringToByteArray(fileContent, EncodingType.ASCII);
Here is the function I'm using to convert the string:
public static byte[] StringToByteArray(string str, EncodingType encodingType)
{
System.Text.Encoding encoding = null;
switch (encodingType)
{
case EncodingType.ASCII:
encoding = new System.Text.ASCIIEncoding();
break;
case EncodingType.Unicode:
encoding = new System.Text.UnicodeEncoding();
break;
case EncodingType.UTF7:
encoding = new System.Text.UTF7Encoding();
break;
case EncodingType.UTF8:
encoding = new System.Text.UTF8Encoding();
break;
}
return encoding.GetBytes(str);
}
public enum EncodingType
{
ASCII,
Unicode,
UTF7,
UTF8
}
It's obvious to me that calling the above function with EncodingType.ASCII is wrong but I am very confused about what would be correct? What is the proper "match" between "Base64" sent from ColdFusion and the way the string should be encoded in .Net?
Please note that all the code "works" but the subsequent retrieval of a file shows it to be scrambled and I'm pretty sure I have the wrong encoding here.
EDIT-update:
I added the enum code previously omitted. I've tried all of these Encoding Types; they all result in "garbage". That is: I have tried each of these variations:
byte[] binData = StringToByteArray(fileContent, EncodingType.ASCII);
byte[] binData = StringToByteArray(fileContent, EncodingType.Unicode);
byte[] binData = StringToByteArray(fileContent, EncodingType.UTF7);
byte[] binData = StringToByteArray(fileContent, EncodingType.UTF8);
None of these work properly. As I read your suggested function, it should be Unicode. Note that I want to return a byte array not a converted string. Still very confused.
ANSWER:
I simply eliminated the enum and the function I wrote called StringToByteArray. Instead I coded the following:
byte[] binData = Convert.FromBase64String(fileContent);
Look at the Convert.FromBase64String() function
Base64 is an encoding scheme that enables you to represent binary data as a series of ASCII characters so that it can be included in text files and e-mail messages in which raw binary data is unacceptable. The below examples show encoding and decoding of unicode strings. Let me know if this is what you wanted,if not I can refind this further for you.
//Encoding
public static string StringToBase64 (string src) {
// Get's byte representation unicode string
byte[] b = Encoding.Unicode.GetBytes(src);
// Returns Base64-encoded string
return Convert.ToBase64String(b);
}
//Decoding
public static string Base64ToString (string src) {
// Decodes Base64-encoded string to a byte array
byte[] b = Convert.FromBase64String(src);
// Returns decoded Unicode string
return Encoding.Unicode.GetString(b);
}