Related
I am currently working on a C# forms application that needs to access a specific cookie on my computer, which I can do perfectly fine. Here's the issue:
Google stores cookies in SQLite, and I've downloaded Sqlite database browser to help me look at these values. What surprises me is that about half of the cookie values shows as empty (including the one I need), even though they are obviously not.
The db file is located at:
C:\Users\%username%\AppData\Local\Google\Chrome\User Data\Default\Cookies
On Chrome I have an addon called "Edit This Cookie" which allows me to directly modify cookies on the website I'm on. This addon can read these cookies, and the web browser can parse values through HTTP when needed for different requests, so they are definitely there - still, the SQLite browser, and my custom code both come to the conclusion that the particular value field is empty.
Why is that?
What is it that somehow prevents the field from being read by certain applications?
I've run into this same problem, and the code below provides a working example for anyone who is interested. All credit to Scherling, as the DPAPI was spot on.
public class ChromeCookieReader
{
public IEnumerable<Tuple<string,string>> ReadCookies(string hostName)
{
if (hostName == null) throw new ArgumentNullException("hostName");
var dbPath = Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData) + #"\Google\Chrome\User Data\Default\Cookies";
if (!System.IO.File.Exists(dbPath)) throw new System.IO.FileNotFoundException("Cant find cookie store",dbPath); // race condition, but i'll risk it
var connectionString = "Data Source=" + dbPath + ";pooling=false";
using (var conn = new System.Data.SQLite.SQLiteConnection(connectionString))
using (var cmd = conn.CreateCommand())
{
var prm = cmd.CreateParameter();
prm.ParameterName = "hostName";
prm.Value = hostName;
cmd.Parameters.Add(prm);
cmd.CommandText = "SELECT name,encrypted_value FROM cookies WHERE host_key = #hostName";
conn.Open();
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
var encryptedData = (byte[]) reader[1];
var decodedData = System.Security.Cryptography.ProtectedData.Unprotect(encryptedData, null, System.Security.Cryptography.DataProtectionScope.CurrentUser);
var plainText = Encoding.ASCII.GetString(decodedData); // Looks like ASCII
yield return Tuple.Create(reader.GetString(0), plainText);
}
}
conn.Close();
}
}
}
Alright, so in case anyone is interested, I found a solution to this problem after alot of trial, error and googling.
Google Chrome cookies DB has 2 columns for storing values: "value" and "encrypted_value", the latter being used when the cookie stored was requested to be encrypted - often the case with certain confidential information and long-time session keys.
After figuring this out, I then needed to find a way to access this key, stored as a Blob value. I found several guides on how to do this, but the one that ended up paying of was: http://www.codeproject.com/Questions/56109/Reading-BLOB-in-Sqlite-using-C-NET-CF-PPC
Simply reading the value is not enough, as it is encrypted. - Google Chrome uses triple DES encryption with the current users password as seed on windows machines. In order to decrypt this in C#, one should use Windows Data Protection API (DPAPI), there are a few guides out there on how to make use of it.
Like Jasper's answer, in a PowerShell script (of course, customize the SQL query to your needs, and the path to your cookies location):
$cookieLocation = 'C:\Users\John\AppData\Local\Google\Chrome\User Data\Default\cookies'
$tempFileName = [System.IO.Path]::GetTempFileName()
"select writefile('$tempFileName', encrypted_value) from cookies where host_key = 'localhost' and path = '/api' and name = 'sessionId';" | sqlite3.exe "$cookieLocation"
$cookieAsEncryptedBytes = Get-Content -Encoding Byte "$tempFileName"
Remove-Item "$tempFileName"
Add-Type -AssemblyName System.Security
$cookieAsBytes = [System.Security.Cryptography.ProtectedData]::Unprotect($cookieAsEncryptedBytes, $null, [System.Security.Cryptography.DataProtectionScope]::CurrentUser)
$cookie = [System.Text.Encoding]::ASCII.GetString($cookieAsBytes)
$cookie
So I wanted to do this without writing to a tempfile every time but also without implementing a separate class as per jasper's solution. Like jasper, I found it easier and quicker to access the System.Data.SQLite.dll available here. It's not as elegant as a separate class, but it's what worked best for me:
Add-Type -AssemblyName System.Security
Add-Type -Path 'C:\Program Files\System.Data.SQLite\2015\bin\x64\System.Data.SQLite.dll'
Function Get-Last-Cookie {
Param(
[Parameter(Mandatory=$True)] $valueName,
[Parameter(Mandatory=$True)] $hostKey,
[Parameter(Mandatory=$True)] $dbDataSource
)
$conn = New-Object -TypeName System.Data.SQLite.SQLiteConnection
$conn.ConnectionString = "Data Source=$dbDataSource"
$conn.Open()
$command = $conn.CreateCommand()
$query = "SELECT encrypted_value FROM cookies WHERE name='$valueName' `
AND host_key='$hostKey' ORDER BY creation_utc DESC LIMIT 1"
$command.CommandText = $query
$adapter = New-Object -TypeName System.Data.SQLite.SQLiteDataAdapter $command
$dataset = New-Object System.Data.DataSet
[void]$adapter.Fill($dataset)
$command.Dispose();
$conn.Close();
$cookieAsEncryptedBytes = $dataset.Tables[0].Rows[0].ItemArray[0]
$cookieAsBytes = [System.Security.Cryptography.ProtectedData]::Unprotect($cookieAsEncryptedBytes, $null, [System.Security.Cryptography.DataProtectionScope]::CurrentUser)
return [System.Text.Encoding]::ASCII.GetString($cookieAsBytes)
}
$localAppDataPath = [Environment]::GetFolderPath([Environment+SpecialFolder]::LocalApplicationData)
$cookieDbPath = 'Google\Chrome\User Data\Default\Cookies'
$dbDataSource = Join-Path -Path $localAppDataPath -ChildPath $cookieDbPath
$plainCookie = Get-Last-Cookie 'acct' '.stackoverflow.com' $dbDataSource
Write-Host $plainCookie
I also found the Add-SqliteAssembly function by halr9000 to be very helpful when it came time to schedule my script in the windows task scheduler and realized that the task scheduler runs the x86 version of PowerShell and thus SQLite rather than the x64 I was using in the console.
The thing is that Google Chrome encrypts the data you need to read, so you have to decrypt it.
First, get a copy of the cookies file. Then read it using SQLite3. After that, get the encrypted bytes. And last, use the code below to decrypt it.
You'll need these Nugets:
using System.IO;
using System.Net;
using System.Data.SQLite;
using Org.BouncyCastle.Crypto.Engines;
using Org.BouncyCastle.Crypto.Modes;
using Org.BouncyCastle.Crypto.Parameters;
using Org.BouncyCastle.Crypto;
using Newtonsoft.Json.Linq;
The code so far:
File.Copy(Environment.GetEnvironmentVariable("APPDATA") + #"/../Local/Google/Chrome/User Data/Default/Cookies", #"./Cookies",true);
SQLiteConnection Cnn = new SQLiteConnection("Data Source=" + #"./Cookies" + ";pooling=false");
Cnn.Open();
SQLiteCommand cmd = new SQLiteCommand("SELECT host_key, name, value, encrypted_value FROM cookies WHERE name='mvrusername' OR name='mvrcookie' OR name='mikuki4'", Cnn);
SQLiteDataReader rdr = cmd.ExecuteReader();
while (rdr.Read())
{
byte[] encryptedData = (byte[])rdr["encrypted_value"];
string encKey = File.ReadAllText(Environment.GetEnvironmentVariable("APPDATA") + #"/../Local/Google/Chrome/User Data/Local State");
encKey = JObject.Parse(encKey)["os_crypt"]["encrypted_key"].ToString();
var decodedKey = System.Security.Cryptography.ProtectedData.Unprotect(Convert.FromBase64String(encKey).Skip(5).ToArray(), null, System.Security.Cryptography.DataProtectionScope.LocalMachine);
const int MAC_BIT_SIZE = 128;
const int NONCE_BIT_SIZE = 96;
using (var cipherStream = new MemoryStream(encryptedData))
using (var cipherReader = new BinaryReader(cipherStream))
{
var nonSecretPayload = cipherReader.ReadBytes(3);
var nonce = cipherReader.ReadBytes(NONCE_BIT_SIZE / 8);
var cipher = new GcmBlockCipher(new AesEngine());
var parameters = new AeadParameters(new KeyParameter(decodedKey), MAC_BIT_SIZE, nonce);
cipher.Init(false, parameters);
var cipherText = cipherReader.ReadBytes(encryptedData.Length);
var plainText = new byte[cipher.GetOutputSize(cipherText.Length)];
try
{
var len = cipher.ProcessBytes(cipherText, 0, cipherText.Length, plainText, 0);
cipher.DoFinal(plainText, len);
}
catch (InvalidCipherTextException)
{
}
string _cookie= Encoding.Default.GetString(plainText);
}
}
// Big thanks to https://stackoverflow.com/a/60611673/6481581 for answering how Chrome 80 and up changed the way cookies are encrypted.
# this powershell scripts exports your cookies to a format curl and wget understand
# Obs ! Each profile has its own cookes file , replace me (ysg ;o) with your win usr name
# aka wget -x --load-cookies cookies.txt http://stackoverflow.com/questions/22532870/encrypted-cookies-in-chrome
$cookieLocation = 'C:\Users\ysg\AppData\Local\Google\Chrome\User Data\Profile 1\Cookies'
$curl_cookies_file="C:\var\ygeo.reports.app.futurice.com.cookies.doc-pub-host.txt"
$tempFileName1 = [System.IO.Path]::GetTempFileName()
$tempFileName2 = [System.IO.Path]::GetTempFileName()
# adjust your filter in the where clause ...
"select writefile('$tempFileName1', encrypted_value) from cookies where host_key = '.futurice.com' ;" | sqlite3.exe "$cookieLocation"
$cookieAsEncryptedBytes = Get-Content -Encoding Byte "$tempFileName1"
Remove-Item "$tempFileName1"
Add-Type -AssemblyName System.Security
$cookieAsBytes = [System.Security.Cryptography.ProtectedData]::Unprotect($cookieAsEncryptedBytes, $null, [System.Security.Cryptography.DataProtectionScope]::CurrentUser)
$cookie = [System.Text.Encoding]::ASCII.GetString($cookieAsBytes)
$unquoted_cookie=$cookie -replace '"', ""
# adjust your filter in the where clause ...
"
select
host_key
, CASE WHEN httponly=0 THEN 'FALSE' ELSE 'TRUE' END
, path
, CASE WHEN secure=0 THEN 'FALSE' ELSE 'TRUE' END
, expires_utc
, name
, '$unquoted_cookie'
from cookies where host_key = '.futurice.com' ;" | sqlite3.exe -separator " " "$cookieLocation" > $curl_cookies_file
Get-ChildItem *.txt | ForEach-Object { (Get-Content $_) | Out-File -Encoding ASCII $_ }
# check the meta data table
#"PRAGMA table_info([cookies]);" | sqlite3.exe "$cookieLocation"
# src: https://github.com/daftano/cookies.txt/blob/master/src/popup.js
#content += escapeForPre(cookie.domain);
#content += "\t";
#content += escapeForPre((!cookie.hostOnly).toString().toUpperCase());
#content += "\t";
#content += escapeForPre(cookie.path);
#content += "\t";
#content += escapeForPre(cookie.secure.toString().toUpperCase());
#content += "\t";
#content += escapeForPre(cookie.expirationDate ? Math.round(cookie.expirationDate) : "0");
#content += "\t";
#content += escapeForPre(cookie.name);
#content += "\t";
#content += escapeForPre(cookie.value);
#content += "\n";
#
#0|creation_utc|INTEGER|1||1
#1|host_key|TEXT|1||0
#2|name|TEXT|1||0
#3|value|TEXT|1||0
#4|path|TEXT|1||0
#5|expires_utc|INTEGER|1||0
#6|secure|INTEGER|1||0
#7|httponly|INTEGER|1||0
#8|last_access_utc|INTEGER|1||0
#9|has_expires|INTEGER|1|1|0
#10|persistent|INTEGER|1|1|0
#11|priority|INTEGER|1|1|0
#12|encrypted_value|BLOB|0|''|0
#13|firstpartyonly|INTEGER|1|0|0
Just set "value" to the cookie you want, "encrypted_value" to NULL and "priority" to 0
So, I have a short query that I want to construct. I'm using a boolean query to specify that the "type" field of the Document matched from the index must be "Idea", and then I have a search string given by a user that may be one or more words. I want to be able to restrict the results programatically for the client to only contain docs in the index that have the Field "type" equal to "index", but I also want their search term to be able to match any word in the search phrase with a word in the result. I think my code below explains what I want exactly.
WhitespaceAnalyzer analyzer = new WhitespaceAnalyzer();
MultiFieldQueryParser parser = new MultiFieldQueryParser(
Version.LUCENE_30, new string[] { "company", "description",
"name", "posterName"},
analyzer);
parser.AllowLeadingWildcard = true;
Lucene.Net.Search.Query query = parser.Parse(searchParam);
BooleanQuery bq = new BooleanQuery();
TermQuery tQuery = new TermQuery(new Lucene.Net.Index.Term("type", "Idea"));
bq.Add(tQuery, Lucene.Net.Search.Occur.MUST);
bq.Add(query, Lucene.Net.Search.Occur.MUST);
The way that I am indexing data is described in a short amount of the pertinent code below:
Document doc = new Document();
doc.Add(new Field("type",
"Idea",
Field.Store.YES,
Field.Index.ANALYZED));
doc.Add(new Field("company",
(_idea.Company==null ?
"Company Not Set for Idea"
: _idea.Company.Name),
Field.Store.YES,
Field.Index.ANALYZED));
doc.Add(new Field("description",
_idea.Description,
Field.Store.YES,
Field.Index.ANALYZED));
doc.Add(new Field("name",
_idea.Name,
Field.Store.YES,Field.Index.ANALYZED));
if (_idea.Poster != null)
{
doc.Add(new Field("posterName",
_idea.Poster.FirstName + " " + _idea.Poster.LastName,
Field.Store.YES, Field.Index.ANALYZED));
}
doc.Add(new Field("ID",
_idea.ID.ToString(), Field.Store.YES,
Field.Index.NOT_ANALYZED));
iWriter.AddDocument(doc);
What I don't understand, is that when I search for a given word that I KNOW exists in the index, it returns no results. Its only if I search with a wildcard like "*" or something that I get any results. What I would think is, if the code does exactly what it says it does for the documentation on a MultiFieldQueryParser, it would return matches if any piece of any field in the parameters of company, description, name ect were to be found in a doc. But it doesn't. For example, in one of the docs, I know I have a name field of "Another Idea". When I search for "Another"/"another"/"Idea"/ ect it should return that particular doc. But it doesn't... it does, however, correctly filter the results by the type.
What do I need to do to get this short code snippet to return matches that I want?
I figured out how to solve this question, and it turns out to be a no brainer (depending on how much you know about lucene and using Visual Studio asp projects, which I'm not that familiar with). This is my first.
Turns out that you can use the BooleanQuery object to add different queries together, and specify how you want them to operate together. Then you can pass the final sum of all queries to the searcher.
Turns out, I just wasn't splitting the objects and creating queries off of them: I have attached the sample solution that works for me below:
StandardAnalyzer analyzer =
new StandardAnalyzer(Version.LUCENE_30);
MultiFieldQueryParser mfqp = new MultiFieldQueryParser(
Version.LUCENE_30, new string[] {"company", "description",
"name", "posterName"},
analyzer);
mfqp.DefaultOperator = MultiFieldQueryParser.OR_OPERATOR;
mfqp.AllowLeadingWildcard = true;
BooleanQuery innerExpr = new BooleanQuery();
foreach (string s in searchParam.Split(new char[] {' '})) {
innerExpr.Add(mfqp.Parse(s), Occur.SHOULD);
}
innerExpr.Add(new WildcardQuery(new Term("company", searchParam)), Occur.SHOULD);
innerExpr.Add(new WildcardQuery(new Term("description", searchParam)), Occur.SHOULD);
innerExpr.Add(new WildcardQuery(new Term("name", searchParam)), Occur.SHOULD);
innerExpr.Add(new WildcardQuery(new Term("posterName", searchParam)), Occur.SHOULD);
TermQuery tQuery = new TermQuery(new Term("type", "Idea"));
//bq.Add(mfqp.Parse(searchParam), Lucene.Net.Search.Occur.MUST);
TopDocs hits = sharedIndex.Search(innerExpr,
new QueryWrapperFilter(tQuery), 1000,
new Sort(SortField.FIELD_DOC));
This entire route wasn't clear to me when I started on this.
One improvement you can make to that solution, in order to accommodate future changes to your index, would be to create a string array variable to hold your field names, e.g.:
string[] allFields = new string[] {"company", "description",
"name", "posterName"};
which in turn will give you a value to put into your parser:
MultiFieldQueryParser mfqp = new MultiFieldQueryParser(
Version.LUCENE_30, allFields, analyzer);
and the ability to iterate through the fields and have a single line to add your wildcard queries:
foreach (string searchField in allFields) {
innerExpr.Add(new WildcardQuery(new Term(searchField, searchParam)), Occur.SHOULD);
}
Then, in the future, you need only add/change/remove field names to the array, and not have to manage your list of queries.
I have a text file that contains around 21 lac entries and I want to insert all these entries into a table. Initially I have created one function in c# that read line by line and insert into table but it takes too much time. Please suggest an efficient way to insert these bulk data and that file is containing TAB(4 spaces) as delimiter.
And that text file also containing some duplicate entries and I don't want to insert those entries.
Load all of your data into a DataTable object and then use SqlBulkCopy to bulk insert them:
DataTable dtData = new DataTable("Data");
// load your data here
using (SqlConnection dbConn = new SqlConnection("db conn string"))
{
dbConn.Open();
using (SqlTransaction dbTrans = dbConn.BeginTransaction())
{
try
{
using (SqlBulkCopy dbBulkCopy = new SqlBulkCopy(dbConn, SqlBulkCopyOptions.Default, dbTrans))
{
dbBulkCopy.DestinationTableName = "intended SQL table name";
dbBulkCopy.WriteToServer(dtData );
}
dbTrans.Commit();
}
catch
{
dbTrans.Rollback();
throw;
}
}
dbConn.Close();
}
I've included the example to wrap this into a SqlTransaction so there will be a full rollback if there's a failure along the way. To get you started, here's a good CodeProject article on loading the delimited data into a DataSet object.
Sanitizing the data before loading
OK, here's how I think your data looks:
CC_FIPS FULL_NAME_ND
AN Xixerella
AN Vila
AN Sornas
AN Soldeu
AN Sispony
... (cut down for brevity)
In this instance you want to create your DataTable like this:
DataTable dtData = new DataTable("Data");
dtData.Columns.Add("CC_FIPS");
dtData.Columns.Add("FULL_NAME_ND");
Then you want to iterate each row (assuming your tab delimited data is separated row-by-row by carriage returns) and check whether this data already exists in the DataTable using the .Select method and if there is a match (i'm checking for BOTH values, it's up to you whether you want to do something else) then don't add it thereby preventing duplicates.
using (FileStream fs = new FileStream("path to your file", FileMode.Open, FileAccess.Read))
{
int rowIndex = 0;
using (StreamReader sr = new StreamReader(fs))
{
string line = string.Empty;
while (!sr.EndOfStream)
{
line = sr.ReadLine();
// use a row index to skip the header row as you don't want to insert CC_FIPS and FULL_NAME_ND
if (rowIndex > 0)
{
// split your data up into a 2-d array tab delimited
string[] parts = line.Split('\t');
// now check whether this data has already been added to the datatable
DataRow[] rows = dtData.Select("CC_FIPS = '" + parts[0] + "' and FULL_NAME_ND = '" + parts[1] + "'");
if (rows.Length == 0)
{
// if there're no rows, then the data doesn't exist so add it
DataRow nr = dtData.NewRow();
nr["CC_FIPS"] = parts[0];
nr["FULL_NAME_ND"] = parts[1];
dtData.Rows.Add(nr);
}
}
rowIndex++;
}
}
}
At the end of this you should have a sanitized DataTable that you can bulk insert. Please note that this code isn't tested, but it's a best guess as to how you should do it. There are many ways this can be done, and probably a lot better than this method (specifically LINQ) - but it's a starting point.
I am using the code below for saving uploaded picture and makigna thumbnail but it saves a filename without the extension to the database, therefore, I get broken links. How can I stop a strongly typed dataset and dataadapter to stop removing the file extension? my nvarchar field has nvarchar(max) so problem is not string length.
I realized my problem was the maxsize in the dataset column, not sql statement parameter, so I fixed it. You may vote to close on this question.
hasTableAdapters.has_actorTableAdapter adp1 = new hasTableAdapters.has_actorTableAdapter();
if (Convert.ToInt16(adp1.UsernameExists(username.Text)) == 0)
{
adp1.Register(username.Text, password.Text,
ishairdresser.Checked, city.Text, address.Text);
string originalfilename = Server.MapPath(" ") + "\\pictures\\" + actorimage.PostedFile.FileName;
string originalrelative = "\\pictures\\" + actorimage.FileName;
actorimage.SaveAs(originalfilename);
string thumbfilename = Server.MapPath(" ") + "\\pictures\\t_" + actorimage.PostedFile.FileName;
string thumbrelative = "\\pictures\\t_" + actorimage.FileName;
Bitmap original = new Bitmap(originalfilename);
Bitmap thumb=(Bitmap)original.GetThumbnailImage(100, 100,
new System.Drawing.Image.GetThumbnailImageAbort(ThumbnailCallback),
IntPtr.Zero);
thumb=(Bitmap)original.Clone(
new Rectangle(new Point(original.Width/2,original.Height/2), new Size(100,100)),
System.Drawing.Imaging.PixelFormat.DontCare);
/*
bmpImage.Clone(cropArea,bmpImage.PixelFormat);
*/
thumb.Save(thumbfilename);
adp1.UpdatePicture(originalrelative, thumbrelative, username.Text);
LoginActor();
Response.Redirect("Default.aspx");
}
}
Looks like the problem is you are using HttpPostedFile.FileName property, which returns fully-qualified file name on the client. So, this code string originalfilename = Server.MapPath(" ") + "\\pictures\\" + actorimage.PostedFile.FileName; generates something like this:
c:\inetpub\pictures\c:\Users\Username\Pictures\image1.jpg
Use FileUpload.FileName property everywhere and you will probably get what you want.
Use this to get Image or file extension :
string Extension = System.IO.Path.GetExtension(FileUpload.FileName);
i have written the code of jsp,servlet for uploading the Doc file in database.
here is my code,but i am getting the error like this==java.io.FileNotFoundException: insert into resume(resume) values(?) (The filename, directory name, or volume label syntax is incorrect) .please help me how to remove this error???
try
{
Class.forName("net.sourceforge.jtds.jdbc.Driver").newInstance();
con=DriverManager.getConnection("jdbc:jtds:sqlserver://W2K8SERVER:1433/career","sa","Alpha#123" );
pst = con.prepareStatement("select * from data1 where email=? and password=?");
pst = con.prepareStatement
("insert into resume(resume) "+ "values(?)");
File re = new File("" + pst);
fis = new FileInputStream(re);
pst.setBinaryStream(3, (InputStream)fis, (int)(re.length()));
pst.setString (1,upload);
//rs = pst.executeQuery();
while (rs.next())
cnt ++;
int s = pst.executeUpdate();
if(s>0) {
System.out.println("Uploaded successfully !");
}
else {
System.out.println("unsucessfull to upload image.");
}
rs.close();
pst.close();
con.close();
}
It is because insert into resume(resume) values(?) is probably not the name of a file on your disk.
pst = con.prepareStatement ("insert into resume(resume) "+ "values(?)");
File re = new File("" + pst);
You are creating a File from ""+ pst, which is the result of toString() being called on a PreparedStatement. Did you put in the "" to get rid of the informative compilation error?
You probably have the real filename in some other variable.
File re = new File(theRealFileNameVariable);
File re = new File("" + pst);
please make sure the "pst" value is a valid file path, apparently the value "insert into resume(resume) values(?)" is not a valid file path
The java.io.File class has four (4) constructors
File(File parent, String child)
File(String pathname)
File(String parent, String child)
File(URI uri)
In your case you are trying to pass to the constructor an object or type java.sql.PreparedStatement that you trying to cast into a String. I don't see how (even if you arrived to convert pst into a string) pst will reference a path to a file. Please go back and read a little bit about java.io.
.