You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
href="https://doi.org/10.1016/j.cognition.2007.05.006">Levy, 2008</a>) which
36
-
assumes that the expected processing cost increases linearly with surprisal,
37
-
and makes no prediction about variance. In the second part of this paper, we
38
-
conduct an empirical study of the relationship between surprisal and reading
39
-
time, using a collection of modern language models to estimate surprisal. We
40
-
find that with better language models, reading time increases superlinearly in
41
-
surprisal, and also that variance increases. These results are consistent with
42
-
the predictions of sampling-based algorithms.},
26
+
abstract = {Words that are more surprising given context take longer to process. However, no incremental parsing algorithm has been shown to directly predict this phenomenon. In this work, we focus on a class of algorithms whose runtime does naturally scale in surprisal\textemdash those that involve repeatedly sampling from the prior. Our first contribution is to show that simple examples of such algorithms predict runtime to increase superlinearly with surprisal, and also predict variance in runtime to increase. These two predictions stand in contrast with literature on surprisal theory (<a href="https://www.aclweb.org/anthology/N01-1021">Hale, 2001</a>; <a href="https://doi.org/10.1016/j.cognition.2007.05.006">Levy, 2008</a>) which assumes that the expected processing cost increases linearly with surprisal, and makes no prediction about variance. In the second part of this paper, we conduct an empirical study of the relationship between surprisal and reading time, using a collection of modern language models to estimate surprisal. We find that with better language models, reading time increases superlinearly in surprisal, and also that variance increases. These results are consistent with the predictions of sampling-based algorithms.},
(2008)</a> explain this variation as arising due to differences
61
-
between varieties in the timing of subject raising, using a split
62
-
phi-probe. This paper revises their approach, using the feature
63
-
gluttony mechanism for Agree developed in <a
64
-
href="https://ling.auf.net/lingbuzz/004224">Coon & Keine
65
-
(2020)</a>, and a split phi-probe in which person probing precedes
66
-
number probing. Within this framework, the observed variation
67
-
can be captured by allowing variability two independent
68
-
parameters: the timing of EPP subject raising, and the visibility
69
-
of a number feature on dative DPs. The proposed mechanism
70
-
describes the variation, including predicting the observed
71
-
optional agreement in certain cases that previous literature had
72
-
struggled to account for, and makes additional predictions about
73
-
the differences between varieties in cases of syncretism within
74
-
the verbal paradigm. An investigation into these predictions
75
-
should allow this already well-studied area of Icelandic grammar
76
-
to continue to be a useful test-case for crosslinguistic
77
-
assumptions about the mechanism of Agree, and the status of dative
78
-
arguments.},
38
+
abstract = {Icelandic dative-nominative constructions exhibit a syntactic hierarchy effect known as the Person Restriction: only third person nominatives may control agreement. In these constructions, there is variation between speakers in the extent to which the verb agrees with the nominative for number. <a href="https://ling.auf.net/lingbuzz/000371">Sigurðsson & Holmberg (2008)</a> explain this variation as arising due to differences between varieties in the timing of subject raising, using a split phi-probe. This paper revises their approach, using the feature gluttony mechanism for Agree developed in <a href="https://ling.auf.net/lingbuzz/004224">Coon & Keine (2020)</a>, and a split phi-probe in which person probing precedes number probing. Within this framework, the observed variation can be captured by allowing variability two independent parameters: the timing of EPP subject raising, and the visibility of a number feature on dative DPs. The proposed mechanism describes the variation, including predicting the observed optional agreement in certain cases that previous literature had struggled to account for, and makes additional predictions about the differences between varieties in cases of syncretism within the verbal paradigm. An investigation into these predictions should allow this already well-studied area of Icelandic grammar to continue to be a useful test-case for crosslinguistic assumptions about the mechanism of Agree, and the status of dative arguments.},
79
39
address = {Somerville, Mass., USA},
80
40
author = {Jacob Louis Hoover},
81
41
booktitle = {Proceedings of the 38th West Coast Conference on Formal Linguistics},
abstract = {Are pairs of words that tend to occur together also likely to
96
-
stand in a linguistic dependency? This empirical question is motivated by a
97
-
long history of literature in cognitive science, psycholinguistics, and NLP.
98
-
In this work we contribute an extensive analysis of the relationship between
99
-
linguistic dependencies and statistical dependence between words. Improving on
100
-
previous work, we introduce the use of large pretrained language models to
101
-
compute contextualized estimates of the pointwise mutual information between
102
-
words (CPMI). For multiple models and languages, we extract dependency trees
103
-
which maximize CPMI, and compare to gold standard linguistic dependencies.
104
-
Overall, we find that CPMI dependencies achieve an unlabelled undirected
105
-
attachment score of at most $\approx 0.5$. While far above chance, and
106
-
consistently above a non-contextualized PMI baseline, this score is generally
107
-
comparable to a simple baseline formed by connecting adjacent words. We
108
-
analyze which kinds of linguistic dependencies are best captured in CPMI
109
-
dependencies, and also find marked differences between the estimates of the
110
-
large pretrained language models, illustrating how their different training
111
-
schemes affect the type of dependencies they capture.},
55
+
abstract = {Are pairs of words that tend to occur together also likely to stand in a linguistic dependency? This empirical question is motivated by a long history of literature in cognitive science, psycholinguistics, and NLP. In this work we contribute an extensive analysis of the relationship between linguistic dependencies and statistical dependence between words. Improving on previous work, we introduce the use of large pretrained language models to compute contextualized estimates of the pointwise mutual information between words (CPMI). For multiple models and languages, we extract dependency trees which maximize CPMI, and compare to gold standard linguistic dependencies. Overall, we find that CPMI dependencies achieve an unlabelled undirected attachment score of at most $\approx 0.5$. While far above chance, and consistently above a non-contextualized PMI baseline, this score is generally comparable to a simple baseline formed by connecting adjacent words. We analyze which kinds of linguistic dependencies are best captured in CPMI dependencies, and also find marked differences between the estimates of the large pretrained language models, illustrating how their different training schemes affect the type of dependencies they capture.},
112
56
address = {Online and Punta Cana, Dominican Republic},
113
57
author = {Hoover, Jacob Louis and Du, Wenyu and Sordoni, Alessandro and O{'}Donnell, Timothy J.},
114
58
booktitle = {Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
0 commit comments