I am using a map to store random string keys to *os.File objects. Users will be uploading a file and I want to hold a reference to the file in a global map so I can delete it later.
I have an http handler to process the upload and at the end, I map a random key from the OS uuidgen to "logBundleFile" which is type *os.File.
var db = map[string]*os.File{}
func uploadHandler(w http.ResponseWriter, r *http.Request) {
r.ParseMultipartForm(5 << 30)
file, handler, err := r.FormFile("file")
if err != nil {
log.Fatalf("Error retrieving the file: %v", err)
return
}
defer file.Close()
logBundleFile, err := ioutil.TempFile("", handler.Filename)
if err != nil {
log.Fatal(err)
}
defer logBundleFile.Close()
fileBytes, err := ioutil.ReadAll(file)
if err != nil {
log.Fatalf("Error reading file: %v", err)
}
logBundleFile.Write(fileBytes)
id, err := exec.Command("uuidgen").Output()
idStr := string(id[:])
//id := "1"
if err != nil {
log.Fatal(err)
}
db[idStr] = logBundleFile
log.Printf("ID: %v Type: %T\n", idStr, idStr)
log.Printf("val: %v Type: %T\n\n", db[idStr], db[idStr])
http.Redirect(w, r, fmt.Sprintf("/%s", idStr), http.StatusMovedPermanently)
}
Once that is done, you get redirected to this sessionHandler. It will check if the ID in the body is valid, i.e, mapped to a *os.File. The "ok" bool is always returning false.
func sessionHandler(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
id := vars["id"]
log.Printf("ID: %v Type: %T\n", id, id)
log.Printf("val: %v Type: %T\n", db[id], db[id])
if val, ok := db[id]; ok {
w.Write([]byte(fmt.Sprintf("Session %s %v", id, val)))
} else {
http.Redirect(w, r, "/", http.StatusMovedPermanently)
}
}
Here is an output from the prints. In the uploadHandler, we can see that we have a string key mapped to a non-nil *os.File.
But in the session handler, the same string key maps to a nil *os.File. I don't know what is going on.
2019/08/27 19:49:49 ID: BA06C157-451E-48B5-85F9-8069D9A4EFCE
Type: string
2019/08/27 19:49:49 val: &{0xc000160120} Type: *os.File
2019/08/27 19:49:49 ID: BA06C157-451E-48B5-85F9-8069D9A4EFCE Type: string
2019/08/27 19:49:49 val: <nil> Type: *os.File
It's because in the uploadHandler, the id variable contains newline. If we take a look closely on the log we can see it. somehow Type: string text is printed in the 2nd line.
2019/08/27 19:49:49 ID: BA06C157-451E-48B5-85F9-8069D9A4EFCE // <-- newline
Type: string
2019/08/27 19:49:49 ID: BA06C157-451E-48B5-85F9-8069D9A4EFCE Type: string
Putting trim operation on the idStr should solve the problem.
idStr := strings.TrimSpace(string(id[:]))
Related
In the following codes, s.getFile gets file from S3 and return a struct of io.ReadCloser and ContentLength.
WriteResultesponse write the file to http.ResponseWriter.
But *reader.ContentLength sometimes is different from actualContentLength.
Any idea why? Thanks
s3Ctx, closeS3 := context.WithTimeout(ctx, xxx) // 1 hour
defer closeS3()
// directly stream result from locations
for _, location := range Locations {
reader, err := s.getFile(s3Ctx, xxx)
// reader is
//struct {
// Data io.ReadCloser
// ContentLength *int64
//}
if err != nil {
return err
}
actualContentLength, err := WriteResultesponse(params.Writer, ResultResponse{
Data: reader.Data,
})
}
// WriteResultResponse streams the result data to the user.
func WriteResultResponse(w http.ResponseWriter, resultResp ResultResponse) (int64, error) {
w.Header().Set("Content-Type", "text/plain")
// resultResp.Data is io.ReadCloser
defer resultResp.Data.Close()
return io.Copy(w, resultResp.Data)
}
UPDATE
How about
if f, ok := params.Writer.(http.Flusher); ok {
f.Flush()
}
?
I receive the contents of a file from a data source in chunks. As and when I receive the chunk I want to send the chunk data to a service using http POST request. And by keeping alive the same http POST connection used for sending the first chunk I want to send the remaining chunks of data.
I came up with the following code snippet to implement something similar.
Server-Side
func handle(w http.ResponseWriter, req *http.Request) {
buf := make([]byte, 256)
var n int
for {
n, err := req.Body.Read(buf)
if n == 0 && err == io.EOF {
break
}
fmt.Printf(string(buf[:n]))
}
fmt.Printf(string(buf[:n]))
fmt.Printf("Transfer Complete")
}
Client-Side
type alphaReader struct {
reader io.Reader
}
func newAlphaReader(reader io.Reader) *alphaReader {
return &alphaReader{reader: reader}
}
func (a *alphaReader) Read(p []byte) (int, error) {
n, err := a.reader.Read(p)
return n, err
}
func (a *alphaReader) Reset(str string) {
a.reader = strings.NewReader(str)
}
func (a *alphaReader) Close() error {
return nil
}
func main() {
tr := http.DefaultTransport
alphareader := newAlphaReader(strings.NewReader("First Chunk"))
client := &http.Client{
Transport: tr,
Timeout: 0,
}
req := &http.Request{
Method: "POST",
URL: &url.URL{
Scheme: "http",
Host: "localhost:8080",
Path: "/upload",
},
ProtoMajor: 1,
ProtoMinor: 1,
ContentLength: -1,
Body: alphareader,
}
fmt.Printf("Doing request\n")
_, err := client.Do(req)
alphareader.Reset("Second Chunk")
fmt.Printf("Done request. Err: %v\n", err)
}
Here I want that when I do alphareader.Reset("Second Chunk"), the string "Second Chunk" should be sent using the POST connection made earlier. But that is not happening. The connection gets closed after sending the First Chunk of data. Also I have not written the Close() method properly which I'm not sure how to implement.
I'm newbie to golang and any suggestions would be greatly helpful regarding the same.
A *strings.Reader returns io.EOF after the initial string has been read and your wrapper does nothing to change that, so it cannot be reused. You're looking for io.Pipe to turn the request body into an io.Writer.
package main
import (
"io"
"net/http"
)
func main() {
pr, pw := io.Pipe()
req, err := http.NewRequest("POST", "http://localhost:8080/upload", pr)
if err != nil {
// TODO: handle error
}
go func() {
defer pw.Close()
if _, err := io.WriteString(pw, "first chunk"); err != nil {
_ = err // TODO: handle error
}
if _, err := io.WriteString(pw, "second chunk"); err != nil {
_ = err // TODO: handle error
}
}()
res, err := http.DefaultClient.Do(req)
if err != nil {
// TODO: handle error
}
res.Body.Close()
}
Also, don't initialize the request using a struct literal. Use one of the constructors instead. In your code you're not setting the Host and Header fields, for instance.
I'm working on some tests in Go and I have spent the past 2 days trying to make it work but I couldn't. My problem is that the test returns 400 even when the user does exist.
This is my getUser function
func (handler *UserHandler) getUser(w http.ResponseWriter, ID int) {
logfile, err := os.OpenFile("events.log", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
if err != nil {
log.Fatalf("Error opening file: %v", err)
}
defer logfile.Close()
log.SetOutput(logfile)
user := db.Fetch(ID)
userJSON, err := json.Marshal(user)
if err != nil {
log.Printf("Error while marshaling the user into JSON: %v", err)
return
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
// userJSON is sent as http Response
w.Write(userJSON)
}
This is my UserHandler
type UserHandler struct{}
func (handle *UserHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
var head string
head, r.URL.Path = ShiftPath(r.URL.Path)
id, err := strconv.Atoi(head)
if err != nil {
http.Error(w, fmt.Sprintf("Invalid user ID %q", head), http.StatusBadRequest)
return
}
switch r.Method {
case "GET":
handle.getUser(w, id)
default:
http.Error(w, "Only GET is allowed", http.StatusMethodNotAllowed)
}
}
func ShiftPath(p string) (head, tail string) {
p = path.Clean("/" + p)
i := strings.Index(p[1:], "/") + 1
if i <= 0 {
return p[1:], "/"
}
return p[1:i], p[i:]
}
And this is my test
func TestGetUser(t *testing.T) {
handler := new(UserHandler)
mux := http.NewServeMux()
mux.HandleFunc("/user/", handler.ServeHTTP)
writer := httptest.NewRecorder()
request, _ := http.NewRequest("GET", "/user/12", nil)
mux.ServeHTTP(writer, request)
if writer.Code != 200 {
t.Errorf("Response code is %v", writer.Code)
}
}
Issue with code ====> id, err := strconv.Atoi(head)
Due to error you see a return and hence you see 400 error.
Have your server code fully functional with valid logic.
Suggestion: Always print or debug line by line. You can find the issue and root cause.
I am using golang and firego for connecting to Firebase. I want to update my data Statusfrom ON to OFF with key IDAgent: 7. This is my Database Structure
Image
Assumption : I don't know child active_chat. How can i update data in active_chat/-Koja8GuFplEN3kjbfPO where IDAgent = 7
I have tried this code
x := map[string]string{"Status": "OFF"}
ref.OrderBy("IDAgent").EqualTo("7").Update(x)
but this code wrong query.
In two ways you can do, as per Firebase doc with firego client library. Drafted answer based on from firego README.md.
Note: You have not provided the complete path of the structure, I have drafted the answer based on screenshot. So update your JSON path accordingly.
Approach 1:
f := firego.New("https://my-firebase-app.firebaseIO.com/active-chat/Koja8GuFpIEN3kjbfPO.json", nil)
x := map[string]string{
"Status": "OFF",
}
if err := f.Update(x); err != nil {
log.Fatal(err)
}
Approach 2:
f := firego.New("https://my-firebase-app.firebaseIO.com", nil)
f = f.Ref("/active-chat/Koja8GuFpIEN3kjbfPO.json")
x := map[string]string{
"Status": "OFF",
}
if err := f.Update(x); err != nil {
log.Fatal(err)
}
Update for 2022:
package main
import (
"context"
"fmt"
"time"
firestore "cloud.google.com/go/firestore"
firebase "firebase.google.com/go"
"google.golang.org/api/option"
)
type (
myDocument struct {
Cars []Car `firestore:"cars"`
carsCount int64 `firestore:"car_count"`
UpdateTime string `firestore:"update_time"`
}
Car struct {
Name string `firestore:"name"`
YearBuilt string `firestore:"year_built"`
}
)
func getFirebaseClient(ctx context.Context) (*firestore.Client, error) {
sa := option.WithCredentialsFile("Path_To_Firebase_Key")
// Initialize firebase app with admin privileges
app, err := firebase.NewApp(ctx, nil, sa)
if err != nil {
err = fmt.Errorf("getFirestoreClient failed: %s", err)
return nil, err
}
// Create client
client, err := app.Firestore(ctx)
if err != nil {
err = fmt.Errorf("failed to connect to firestore: %v", err)
return nil, err
}
return client, nil
}
func main() {
// Create context
ctx := context.Background()
// Get firebase client
client, err := getFirebaseClient(ctx)
if err != nil {
panic(err)
}
// Create car struct
newCar := Car{
"Volvo_Series1",
"1920",
}
// Update time
newTime := time.Now().UTC().Format("Monday, 01-02-2006 15:04:05")
// Updates to document
updates := []firestore.Update{
{Path: "cars", Value: firestore.ArrayUnion(newCar)},
{Path: "car_count", Value: firestore.Increment(1)},
{Path: "update_date", Value: newTime},
}
// OPTION A)
// Create collection reference
collectionRef := client.Collection("cars")
// Create document reference
docRef := collectionRef.Doc("12345")
// Update document
_, err = docRef.Update(ctx, updates)
if err != nil {
err := fmt.Errorf("failed updating document: %s from %s collection %v", docRef.ID, docRef.Parent.ID, err)
panic(err)
}
// OPTION B)
_, err = client.Collection("cars").Doc("12345").Update(ctx, updates)
if err != nil {
err := fmt.Errorf("failed updating document: %s from %s collection %v", docRef.ID, docRef.Parent.ID, err)
panic(err)
}
}
I am trying to learn Go with GAE.
I have created 2 handlers. One for saving an object to datastore and the other retrieve it and output to screen. The problem is that when i retrieve the UserAccount object from datastore, every values inside the object are gone.
Any help would be appreciate.
Output:
a/c count: 2
val: core.UserAccount{idString:"", deviceId:""}
val: core.UserAccount{idString:"", deviceId:""}
type UserAccount struct {
idString string
deviceId string
}
func create_account(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
idstr := "ABCDEFGH"
devId := r.FormValue("deviceId")
newAccount := UserAccount{ idString: idstr, deviceId: devId,}
key := datastore.NewIncompleteKey(c, "UserAccount", nil)
_, err := datastore.Put(c, key, &newAccount)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
fmt.Fprintf(w, "val: %#v \n", newAccount)
}
func get_info(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
q := datastore.NewQuery("UserAccount")
accounts := make([]UserAccount, 0, 10)
if _, err := q.GetAll(c, &accounts); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
fmt.Fprintf(w, "a/c count: %v \n", len(accounts))
for i := 0; i < len(accounts); i++ {
fmt.Fprintf(w, "val: %#v \n", accounts[i])
}
}
If the datastore API uses reflection, which I presume it does, it cannot access struct fields that aren't exported, i.e. field names that do not begin with a capital letter.
Export them and it should work.