Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ How to release a new version:

## [Unreleased]

### Added
- package `http/param`: can parse form data into embedded structs.

## [0.8.0] - 2024-11-14
### Added
- package `http/param`: can parse into embedded structs.
Expand Down
68 changes: 62 additions & 6 deletions http/param/param.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package param

import (
"encoding"
"errors"
"fmt"
"net/http"
"reflect"
Expand All @@ -11,8 +12,10 @@ import (

const (
defaultTagName = "param"
defaultMaxMemory = 32 << 20 // 32 MB
queryTagValuePrefix = "query"
pathTagValuePrefix = "path"
formTagValuePrefix = "form"
)

// TagResolver is a function that decides from a field tag what parameter should be searched.
Expand All @@ -34,6 +37,13 @@ func TagNameResolver(tagName string) TagResolver {
// PathParamFunc is a function that returns value of specified http path parameter.
type PathParamFunc func(r *http.Request, key string) string

// FormParamFunc is a function that returns value of specified form parameter.
type FormParamFunc func(r *http.Request, key string) string

func DefaultFormParamFunc(r *http.Request, key string) string {
return r.PostFormValue(key)
}

// Parser can Parse query and path parameters from http.Request into a struct.
// Fields struct have to be tagged such that either QueryParamTagResolver or PathParamTagResolver returns
// valid parameter name from the provided tag.
Expand All @@ -43,6 +53,7 @@ type PathParamFunc func(r *http.Request, key string) string
type Parser struct {
ParamTagResolver TagResolver
PathParamFunc PathParamFunc
FormParamFunc FormParamFunc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove

I don't think it makes sense to have any customisation possible here - the default go std parser should be always called. In contrast, the PathParam func is not handled by go std lib, and the PathParamFunc allows caller to configure to their particular path routing library (e.g. chi) to extract the path placeholder/variable.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the note. I agree that likely every func would use just the default, but some might still benefit from interecepting the parsing somehow, for example with a logging mechanism, prefixing of the key, etc. It would perhaps also make sense to make these functions return error for additional validation, but that's out of the scope of this PR.

Anyway, I don't see any harm in having this option here so I don't see a reason for removal, especially with the default being also implemented in the lib, which somewhat benefits the dev experience.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • I would lean towards keeping the library simple and not providing options that are not obvious what they are for
  • the implementation in this PR is a bit weird how it does some part (ParseForm or now ParseMultipartForm) in the "hardcoded" part, but then allow reading the populated fields/form values in the custom FormParamFunc
  • adding the func for form but not for query is inconsistent, maybe there could be a more general way that handles all types of "param", if it is needed for something at all.
  • I'd argue that logging should happen on different layer, instead of this library providing a way to completely change the behaviour of one of the param types. Or as in above bullet point, the library should provide a more obvious way to intercept the parsed params (of all types if possible), without possibility to "break" the core functionality of this library

Yes, there is a possibility to have everything somehow customisable/configurable. I think we should not attempt to create such configurability without a need in some projects using this library. And if this need arises, I think it would make more sense to handle the issue in a thought out, thorough approach, rather than a side effect of one of the implementation of one of the param types.

That being said it is somewhat minor issue, PR looks good to me either way, after the minor comment I added.

}

// DefaultParser returns query and path parameter Parser with intended struct tags
Expand All @@ -51,6 +62,7 @@ func DefaultParser() Parser {
return Parser{
ParamTagResolver: TagNameResolver(defaultTagName),
PathParamFunc: nil, // keep nil, as there is no sensible default of how to get value of path parameter
FormParamFunc: DefaultFormParamFunc,
}
}

Expand All @@ -61,6 +73,13 @@ func (p Parser) WithPathParamFunc(f PathParamFunc) Parser {
return p
}

// WithFormParamFunc returns a copy of Parser with set function for getting form parameters from http.Request.
// For more see Parser description.
func (p Parser) WithFormParamFunc(f FormParamFunc) Parser {
p.FormParamFunc = f
return p
}

// Parse accepts the request and a pointer to struct with its fields tagged with appropriate tags set in Parser.
// Such tagged fields must be in top level struct, or in exported struct embedded in top-level struct.
// All such tagged fields are assigned the respective parameter from the actual request.
Expand Down Expand Up @@ -113,6 +132,7 @@ type paramType int
const (
paramTypeQuery paramType = iota
paramTypePath
paramTypeForm
)

type taggedFieldIndexPath struct {
Expand All @@ -139,21 +159,25 @@ func (p Parser) findTaggedIndexPaths(typ reflect.Type, currentNestingIndexPath [
}
tag := typeField.Tag
pathParamName, okPath := p.resolvePath(tag)
formParamName, okForm := p.resolveForm(tag)
queryParamName, okQuery := p.resolveQuery(tag)

newPath := append(append([]int{}, currentNestingIndexPath...), i)
if okPath {
newPath := make([]int, 0, len(currentNestingIndexPath)+1)
newPath = append(newPath, currentNestingIndexPath...)
newPath = append(newPath, i)
paths = append(paths, taggedFieldIndexPath{
paramType: paramTypePath,
paramName: pathParamName,
indexPath: newPath,
})
}
if okForm {
paths = append(paths, taggedFieldIndexPath{
paramType: paramTypeForm,
paramName: formParamName,
indexPath: newPath,
})
}
if okQuery {
newPath := make([]int, 0, len(currentNestingIndexPath)+1)
newPath = append(newPath, currentNestingIndexPath...)
newPath = append(newPath, i)
paths = append(paths, taggedFieldIndexPath{
paramType: paramTypeQuery,
paramName: queryParamName,
Expand Down Expand Up @@ -194,6 +218,11 @@ func (p Parser) parseParam(r *http.Request, path taggedFieldIndexPath) error {
if err != nil {
return err
}
case paramTypeForm:
err := p.parseFormParam(r, path.paramName, path.destValue)
if err != nil {
return err
}
case paramTypeQuery:
err := p.parseQueryParam(r, path.paramName, path.destValue)
if err != nil {
Expand All @@ -217,6 +246,29 @@ func (p Parser) parsePathParam(r *http.Request, paramName string, v reflect.Valu
return nil
}

func (p Parser) parseFormParam(r *http.Request, paramName string, v reflect.Value) error {
if r.Method != http.MethodPost && r.Method != http.MethodPut && r.Method != http.MethodPatch {
return fmt.Errorf("struct's field was tagged for parsing the form parameter (%s) but request method is not POST, PUT or PATCH", paramName)
}
if err := r.ParseMultipartForm(defaultMaxMemory); err != nil {
if !errors.Is(err, http.ErrNotMultipart) {
return fmt.Errorf("parsing multipart form: %w", err)
}
// Try to parse regular form if not multipart form.
if err := r.ParseForm(); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is called from the ParseMultipartForm, so it is unnecesary to have it also here.

return fmt.Errorf("parsing form: %w", err)
}
}
paramValue := p.FormParamFunc(r, paramName)
if paramValue != "" {
err := unmarshalValue(paramValue, v)
if err != nil {
return fmt.Errorf("unmarshaling form parameter %s: %w", paramName, err)
}
}
return nil
}

func (p Parser) parseQueryParam(r *http.Request, paramName string, v reflect.Value) error {
query := r.URL.Query()
if values, ok := query[paramName]; ok && len(values) > 0 {
Expand Down Expand Up @@ -331,6 +383,10 @@ func (p Parser) resolvePath(fieldTag reflect.StructTag) (string, bool) {
return p.resolveTagWithModifier(fieldTag, pathTagValuePrefix)
}

func (p Parser) resolveForm(fieldTag reflect.StructTag) (string, bool) {
return p.resolveTagWithModifier(fieldTag, formTagValuePrefix)
}

func (p Parser) resolveQuery(fieldTag reflect.StructTag) (string, bool) {
return p.resolveTagWithModifier(fieldTag, queryTagValuePrefix)
}
66 changes: 66 additions & 0 deletions http/param/param_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,72 @@ func TestParser_Parse_PathParam_FuncNotDefinedError(t *testing.T) {
assert.Error(t, err)
}

type structWithFormParams struct {
Subject string `param:"form=subject"`
Amount *int `param:"form=amount"`
Object *maybeShinyObject `param:"form=object"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider also a test where optional (pointer) fields are not present in the request (similar to TestParser_Parse_QueryParam_Pointers - no params

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, added.

Nothing string `param:"form=nothing"`
}

func TestParser_Parse_FormParam(t *testing.T) {
r := chi.NewRouter()
p := DefaultParser()
result := structWithFormParams{
Nothing: "should be replaced",
}
expected := structWithFormParams{
Subject: "world",
Amount: ptr(69),
Object: &maybeShinyObject{
IsShiny: true,
Object: "apples",
},
Nothing: "",
}
var parseError error
r.Post("/hello/objects", func(_ http.ResponseWriter, r *http.Request) {
parseError = p.Parse(r, &result)
})

urlEncodedBodyContent := "subject=world&amount=69&object=shiny-apples&nothing="
urlEncodedBody := strings.NewReader(urlEncodedBodyContent)
req := httptest.NewRequest(http.MethodPost, "https://test.com/hello/objects", urlEncodedBody)
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
r.ServeHTTP(httptest.NewRecorder(), req)

require.NoError(t, parseError)
assert.Equal(t, expected, result)
}

func TestParser_Parse_FormParam_NoParams(t *testing.T) {
p := DefaultParser()
result := structWithFormParams{
Subject: "should be replaced",
Amount: ptr(123), // should be zeroed out
Nothing: "should be replaced",
}
expected := structWithFormParams{
Subject: "",
Amount: nil,
Object: nil,
Nothing: "",
}
var parseError error

r := chi.NewRouter()
r.Post("/hello/objects", func(_ http.ResponseWriter, r *http.Request) {
parseError = p.Parse(r, &result)
})

// Empty form body
req := httptest.NewRequest(http.MethodPost, "https://test.com/hello/objects", strings.NewReader(""))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
r.ServeHTTP(httptest.NewRecorder(), req)

require.NoError(t, parseError)
assert.Equal(t, expected, result)
}

type otherFieldsStruct struct {
Q string `param:"query=q"`
Other string `json:"other"`
Expand Down