RecursionError when Self-Referential Macros #72
Description
I met a RecursionError when running this project on pcre2
git clone https://github.com/PCRE2Project/pcre2 pcre
pcpp ./pcre/src/pcre2_context.c -I ./pcre/ -I /usr/include
An fatal error occurs:
*** EXPAND MACROS in PCRE2_SPTR expanding_from= []
*** EXPAND MACROS in PCRE2_SUFFIX(PCRE2_SPTR) expanding_from= ['PCRE2_SPTR']
*** EXPAND MACROS in PCRE2_SPTR expanding_from= []
*** EXPAND MACROS in PCRE2_SUFFIX(PCRE2_SPTR) expanding_from= ['PCRE2_SPTR']
*** EXPAND MACROS in PCRE2_SPTR expanding_from= []
*** EXPAND MACROS in PCRE2_SUFFIX(PCRE2_SPTR) expanding_from= ['PCRE2_SPTR']
*** EXPAND MACROS in PCRE2_SPTR expanding_from= []
*** EXPAND MACROS in PCRE2_SUFFIX(PCRE2_SPTR) expanding_from= ['PCRE2_SPTR']
*** EXPAND MACROS in PCRE2_SPTR expanding_from= []
*** EXPAND MACROS in PCRE2_SUFFIX(PCRE2_SPTR) expanding_from= ['PCRE2_SPTR']
*** EXPAND MACROS in PCRE2_SPTR expanding_from= []
Traceback (most recent call last):
File "/root/miniconda3/envs/arch/lib/python3.10/site-packages/pcpp/pcmd.py", line 140, in __init__
self.write(self.args.output)
File "/root/miniconda3/envs/arch/lib/python3.10/site-packages/pcpp/preprocessor.py", line 1322, in write
tok = self.token()
File "/root/miniconda3/envs/arch/lib/python3.10/site-packages/pcpp/preprocessor.py", line 1303, in token
tok = next(self.parser)
File "/root/miniconda3/envs/arch/lib/python3.10/site-packages/pcpp/preprocessor.py", line 895, in parsegen
for tok in self.include(args, x):
File "/root/miniconda3/envs/arch/lib/python3.10/site-packages/pcpp/preprocessor.py", line 1167, in include
for tok in self.parsegen(data,filename,fulliname):
File "/root/miniconda3/envs/arch/lib/python3.10/site-packages/pcpp/preprocessor.py", line 895, in parsegen
for tok in self.include(args, x):
File "/root/miniconda3/envs/arch/lib/python3.10/site-packages/pcpp/preprocessor.py", line 1167, in include
for tok in self.parsegen(data,filename,fulliname):
File "/root/miniconda3/envs/arch/lib/python3.10/site-packages/pcpp/preprocessor.py", line 903, in parsegen
for tok in self.expand_macros(chunk):
File "/root/miniconda3/envs/arch/lib/python3.10/site-packages/pcpp/preprocessor.py", line 580, in expand_macros
ex = self.expand_macros(rep, expanding_from + [t.value])
File "/root/miniconda3/envs/arch/lib/python3.10/site-packages/pcpp/preprocessor.py", line 580, in expand_macros
ex = self.expand_macros(rep, expanding_from + [t.value])
RecursionError: maximum recursion depth exceeded while calling a Python object
None
INTERNAL PREPROCESSOR ERROR AT AROUND ../../../usr/include/pcre2.h:917, FATALLY EXITING NOWI tracked the code and it was caused by :
https://github.com/PCRE2Project/pcre2/blob/master/src/pcre2.h.in#L806, the self referential macros was used.
I made a quick fix in expand_macros to prevent infante recursion:
def expand_macros(self,tokens,expanding_from=[]):
"""Given a list of tokens, this function performs macro expansion."""
# Each token needs to track from which macros it has been expanded from to prevent recursion
for tok in tokens:
if not hasattr(tok, 'expanded_from'):
tok.expanded_from = []
if len(expanding_from) == 1 and tok.value == expanding_from[0]:
return tokens
i = 0Not sure whether this is the correct, hope this issue can be addressed on master.