fix: OSD: adjust horizon rules #86
Conversation
…epths, and variants in depth units
| @@ -94,6 +94,29 @@ | |||
| "topography": "NA", | |||
| "narrative": "Ap -- 0 to 10 inches; dark reddish brown (5YR 3/3) silt loam," | |||
| }, | |||
| { | |||
| "name": "struc", | |||
There was a problem hiding this comment.
multiline typical pedon statements RE: #83
There was a problem hiding this comment.
This issue will be addressed by concatenating multiline horizon descriptions
brownag
force-pushed
the
old-o1
branch
2 times, most recently
from
8d6740d to
1ebf695
Compare
|
These are some of the key test cases I have been working with. Reviewing the latest changes now for any unexpected results.
|
|
DISTIN soil series has two profiles, very interesting: https://soilseries.sc.egov.usda.gov/OSD_Docs/D/DISTIN.html Now we parse both but are also picking up part 1/part 2 lines and lost the inch units. Possibly we need to further refine which lines we get the units for. |
inst/extdata/OSD/D/DISTIN.json
Outdated
| { | ||
| "name": "Oe", | ||
| "top": 0, | ||
| "bottom": 3, | ||
| "bottom": 1, |
There was a problem hiding this comment.
not detecting inch units due to the first "horizon" not having depth data
| @@ -140,6 +140,29 @@ | |||
| "topography": "NA", | |||
| "narrative": "Bw2--9 to 25 inches; reddish brown (5YR 5/3) clay, reddish brown" | |||
| }, | |||
| { | |||
There was a problem hiding this comment.
multiline typical pedon #83
inst/extdata/OSD/F/FROSTCIRCLE.json
Outdated
| @@ -71,10 +71,33 @@ | |||
| ], | |||
| "HORIZONS": [ | |||
| [ | |||
| { | |||
| "name": "I", | |||
inst/extdata/OSD/G/GRASSVALLEY.json
Outdated
| @@ -186,6 +186,29 @@ | |||
| "topography": "NA", | |||
| "narrative": "BC--28 to 37 inches; pink (5YR 7/3) clay, reddish brown (5YR" | |||
| }, | |||
| { | |||
| "name": "NA", | |||
inst/extdata/OSD/K/KUSKOKWIM.json
Outdated
| @@ -70,6 +70,76 @@ | |||
| ] | |||
| ], | |||
| "HORIZONS": [ | |||
| {} | |||
| [ | |||
There was a problem hiding this comment.
multiline OSD format, #83, previously no horizons now gets some horizons parsing (poorly)
| "eff_class": "NA", | ||
| "distinctness": "abrupt", | ||
| "topography": "smooth", | ||
| "narrative": "Oi--0 TO 1 inch; forest litter of needles, twigs and leaves; abrupt smooth boundary. (0 to 2 inches thick)" |
There was a problem hiding this comment.
capital "TO" missing bottom depth and units
| "eff_class": "NA", | ||
| "distinctness": "abrupt", | ||
| "topography": "smooth", | ||
| "narrative": "A11--0 to 3; Light brownish gray (10YR 6/2) loamy sand, dark grayish brown (10YR 4/2) moist; weak fine granular structure; soft, very friable, nonsticky, nonplastic; many very fine roots; many very fine interstitial pores; medium acid (pH 5.8); abrupt smooth boundary. (2 to 4 inches thick)" |
There was a problem hiding this comment.
missing units in first horizon
| @@ -70,6 +70,122 @@ | |||
| ] | |||
| ], | |||
| "HORIZONS": [ | |||
| {} | |||
| [ | |||
There was a problem hiding this comment.
no horizons have units on depth (except for thickness which implies inches); previously no horizons parsed
| "eff_class": "NA", | ||
| "distinctness": "gradual", | ||
| "topography": "wavy", | ||
| "narrative": "Bnss--56 o 91 centimeters (22 to 36 inches); dark grayish brown (10YR 4/2) clay, very dark grayish brown (10YR 3/2) moist; strong very coarse prismatic structure; extremely hard, friable, very sticky and very plastic; many very fine roots, mostly on ped faces; common very fine tubular pores; cracks 1 to 5 millimeters wide; ESP is 25.8; EC is 17.7 ds/m; few pressure faces and few intersecting slickensides; neutral (pH 6.8); gradual wavy boundary. (30 to 40 centimeters; 12 to 16 inches thick)." |
There was a problem hiding this comment.
was missing Bnss, now just missing bottom depth, missing "to"
| "eff_class": "NA", | ||
| "distinctness": "abrupt", | ||
| "topography": "smooth", | ||
| "narrative": "A11--0- to 1-inch; light brownish gray (10YR 6/2) silt loam, darkk gray (10YR 4/1) moist; vesicular crust; slightly hard, very friable, sticky and plastic; few fine roots; many very fine vesicular pores; medium acid (pH 6.0); abrupt smooth boundary. (1/2- to 1-inches thick)" |
There was a problem hiding this comment.
was missing A11 horizon, now missing bottom depth due to extra dash
| "eff_class": "NA", | ||
| "distinctness": "clear", | ||
| "topography": "smooth", | ||
| "narrative": "A1--0 to 2 1/2\" inches; light gray (10YR 6/1) loam, dark grayish brown (10YR 4/2) moist; strong very thin platy structure; soft, very friable, slightly sticky, slightly plastic; many very fine and few fine roots; many very fine vesicular pores and common tubular pores; strongly acid (pH 5.3); clear smooth boundary. (2 to 6 inches thick)" |
There was a problem hiding this comment.
was missing A1and A2 horizons, now missing units because of extra "
| "eff_class": "NA", | ||
| "distinctness": "clear", | ||
| "topography": "smooth", | ||
| "narrative": "Al--0 to 10\";Dark graylsh brown (10YR 4/2) cobbly loam, very dark brown (10YR 2/2) molst; moderate flne granular structure; soft, frlable, sllghtly stlcky, slightly plastlc; common flne, medlum, and few coarse roots; few flne tubular pores; slightly acid (pH 6.1); clear smooth boundary. (8 to 12 inches thick)" |
There was a problem hiding this comment.
missing units due to use of "
| "eff_class": "NA", | ||
| "distinctness": "clear", | ||
| "topography": "smooth", | ||
| "narrative": "A-- to 3 inches; brown (10YR 5/3) sandy loam, brown (10YR 4/3) moist; weak thin platy structure; soft, very friable, nonsticky and nonplastic; few fine roots; common fine interstitial and few fine and very fine tubular pores; 10 percent gravel; slightly calcareous; moderately alkaline (pH 8.0); clear smooth boundary. (2 to 5 inches thick)" |
There was a problem hiding this comment.
previously missing A horizon; A horizon narrative is missing top depth, so bottom depth is parsed as top and bottom as NA
| "eff_class": "NA", | ||
| "distinctness": "clear", | ||
| "topography": "wavy", | ||
| "narrative": "C2--26 t0 29 inches--grayish brown (10YR 5/2) very fine sandy loam, very dark grayish brown (10YR 3/2) moist; weak thick platy structure; soft, very friable; few medium and very fine roots; common fine interstitial pores; strongly calcareous; moderately alkaline (pH 8.0); clear wavy boundary. (0 to 4 inches thick)" |
There was a problem hiding this comment.
previously missing C2, now C2 missing bottom depth "t0"
| "eff_class": "NA", | ||
| "distinctness": "NA", | ||
| "topography": "NA", | ||
| "narrative": "2C--20 TO 60 INCHES; strong brown (7.5YR 5/6) extremely gravelly loamy coarse sand; single grain; about 80 percent coarse fragments by volume; very strongly acid." |
| "eff_class": "NA", | ||
| "distinctness": "clear", | ||
| "topography": "wavy", | ||
| "narrative": "Ap--0 to 8 degrees; dark grayish brown (10YR 4/2) silt loam, very dark grayish brown (10YR 3/2) moist; weak fine granular structure; slightly hard, friable, slightly sticky and slightly plastic; common fine roots; many fine and common medium pores; slightly acid (pH 6.1); clear wavy boundary. (6 to 10 inches thick)" |
There was a problem hiding this comment.
Lol. I have never seen horizon depths measured in degrees before. Now units parse wrong because the first horizon was missing previously
| @@ -74,7 +74,7 @@ | |||
| { | |||
| "name": "A1", | |||
| "top": 0, | |||
| "bottom": "NA", | |||
| "bottom": 8, | |||
There was a problem hiding this comment.
now units are parsed correctly
inst/extdata/OSD/N/NORDIC.json
Outdated
| @@ -231,6 +231,29 @@ | |||
| "distinctness": "NA", | |||
| "topography": "NA", | |||
| "narrative": "C--58 to 70 inches, brown (7.5YR 5/4) very cobbly loam, light yellowish brown (lOYR 6/4) dry; weak fine and medium subangular blocky structure; slightly hard, friable, slightly sticky and slightly plastic; few fine roots; 25 percent pebbles, 45 percent cobbles; slightly acid (pH 6.1)." | |||
| }, | |||
| { | |||
| "name": "nvrth", | |||
There was a problem hiding this comment.
this OSD has messed up formatting/encoding
inst/extdata/OSD/R/RUSSIAN.json
Outdated
| "name": "0", | ||
| "top": 20, | ||
| "name": "Ap", | ||
| "top": 0, | ||
| "bottom": "NA", |
There was a problem hiding this comment.
The -- formatting and "to" is not right in first horizon, needs fixing in OSD
| "eff_class": "NA", | ||
| "distinctness": "clear", | ||
| "topography": "smooth", | ||
| "narrative": "C5--37 to 42 degrees; pinkish white (7.5YR 8/2) gravelly loam, light brown (7.5YR 6/3) moist; moderate medium platy structure; very hard, friable, slightly sticky, plastic; few fine and medium roots; few fine pores; 35 percent gravel; very strongly calcareous, lime is disseminated and accumulated in soft rounded accretions; strongly alkaline (pH 8.6); clear smooth boundary. (O to 6 inches thick)" |
There was a problem hiding this comment.
another one with units in degrees (units parsed correctly based on first horizon)
| "eff_class": "NA", | ||
| "distinctness": "NA", | ||
| "topography": "NA", | ||
| "narrative": "01--2 3/4 inches to 3/4 inch; needles, leaves, bark, and wood fragments." |
There was a problem hiding this comment.
Need to add handling for 3/4, 3/8 to fractional depths
inst/extdata/OSD/S/SNAKE.json
Outdated
| "eff_class": "NA", | ||
| "distinctness": "clear", | ||
| "topography": "smooth", | ||
| "narrative": "Alca 0-8\"--Grayish brown (1OYR 5/2) (light brownish gray 1OYR 6/2, crushed) heavy silt loam, very dark grayish brown (1OYR 3/2) (dark grayish brown 1OYR 4/2, crushed) moist; moderate fine and very fine granular structure but weak medium platy in upper 3 inches; slightly hard, friable, sticky, slightly plastic; many very fine to medium roots; many very fine interstitial and tubular pores; strongly calcareous; strongly alkaline (pH 9.0); clear smooth boundary. (6 to 12 inches thick)" |
There was a problem hiding this comment.
no horizon designation separator, units not parsed due to "
inst/extdata/OSD/S/SUGARDEE.json
Outdated
| "eff_class": "NA", | ||
| "distinctness": "NA", | ||
| "topography": "NA", | ||
| "narrative": "2 - Sugardee Series" |
There was a problem hiding this comment.
extra line in middle of typical pedon
inst/extdata/OSD/S/SWILLNA.json
Outdated
| "eff_class": "NA", | ||
| "distinctness": "NA", | ||
| "topography": "NA", | ||
| "narrative": "Part I -- Sixty percent intermounds" |
inst/extdata/OSD/T/TENEX.json
Outdated
| [ | ||
| { | ||
| "name": "NA", | ||
| "top": 0.5, |
There was a problem hiding this comment.
no separator, inch units using "
inst/extdata/OSD/T/TEW.json
Outdated
| "eff_class": "NA", | ||
| "distinctness": "abrupt", | ||
| "topography": "smooth", | ||
| "narrative": "Apca 0-6\"--Very dark gray (2.5Y 3/1) and very dark grayish brown (1OYR 3/2) loam; weak very fine granular structure; very friable, slightly sticky, slightly plastic; many fine and very fine roots; slightly calcareous; moderately alkaline; abrupt smooth boundary (5 to 15 inches thick.)" |
There was a problem hiding this comment.
no separator, inches with "
inst/extdata/OSD/T/TRACYLEE.json
Outdated
| "eff_class": "NA", | ||
| "distinctness": "NA", | ||
| "topography": "NA", | ||
| "narrative": "2 - Tracylee Series" |
There was a problem hiding this comment.
extra line in typical pedon
inst/extdata/OSD/T/TRAPPER.json
Outdated
| {} | ||
| [ | ||
| { | ||
| "name": "l", |
There was a problem hiding this comment.
no separator, OCR errors
| { | ||
| "name": "Oi", | ||
| "top": 11, | ||
| "bottom": 10, |
There was a problem hiding this comment.
units not parsed, "10-1/2" vs "10 1/2"
| "eff_class": "NA", | ||
| "distinctness": "NA", | ||
| "topography": "NA", | ||
| "narrative": "0e--l-1/2 inches to 0; needles, twigs and bark chips." |
There was a problem hiding this comment.
"l-1/2 inches" and first horizon units not parsing
| "eff_class": "NA", | ||
| "distinctness": "gradual", | ||
| "topography": "wavy", | ||
| "narrative": "Bt1--8 to 18 soils; mottled brown (7.5YR 5/4), red (2.5YR 4/6) and light brownish gray (2.5Y 6/2) clay, brown (7.5YR 4/4), dark red (2.5YR 3/6) and grayish brown (2.5Y 5/2) moist; mottles are many fine and medium and distinct; weak medium prismatic structure breaking to strong medium and coarse angular blocky structure; extremely hard, very firm, sticky and very plastic; plentiful fine, medium and coarse roots; few fine tubular pores; common pressure faces; continuous moderately thick clay films on ped faces; few gravel and cobbles; moderately acid (pH 5.6); gradual wavy boundary. (8 to 15 inches thick.)" |
inst/extdata/OSD/W/WILLAKENZIE.json
Outdated
| @@ -185,6 +185,29 @@ | |||
| "distinctness": "NA", | |||
| "topography": "NA", | |||
| "narrative": "Cr---32 inches; weakly cemented tuffaceous sandstone. & Typical pedon relocated based on field observations and documentation from several counties in the MLRA 2 Willamette Valley update area to reflect a modal concept for this soil which has brown hues, active cation exchange activity class, medium to moderately fine textures in the argillic horizon, and dominantly occurs on the Eugene and Spencer Formation geology along the margins of the Willamette. Willakenzie soils occur on the Brateng (low hills) and Dolph (foothills) geomorphic surfaces." | |||
There was a problem hiding this comment.
this is running afoul of the duplicate section combination code, needs to be fixed in new issue
…parsed horizon result
Some ideas for fixing old-style O horizons for #85