Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StackOverflowError when reading a file with a large amount of lines #11

Open
jzaratei opened this issue Jan 8, 2021 · 2 comments
Open

Comments

@jzaratei
Copy link

jzaratei commented Jan 8, 2021

Summary
I'm implementing a LineReader as in the example [0] with a default padder. But, sometimes, I'm getting an StackOverflowError when the file has more than 50 lines aprox.

java.lang.StackOverflowError
	at java.base/java.util.regex.Pattern$Curly.match(Pattern.java:4379)
	at java.base/java.util.regex.Pattern$GroupHead.match(Pattern.java:4804)
	at java.base/java.util.regex.Pattern$GroupTail.match(Pattern.java:4863)
	at java.base/java.util.regex.Pattern$Curly.match0(Pattern.java:4392)
	at java.base/java.util.regex.Pattern$Curly.match(Pattern.java:4379)
	at java.base/java.util.regex.Pattern$GroupHead.match(Pattern.java:4804)
	at java.base/java.util.regex.Pattern$GroupTail.match(Pattern.java:4863)
	at java.base/java.util.regex.Pattern$Curly.match0(Pattern.java:4392)
	at java.base/java.util.regex.Pattern$Curly.match(Pattern.java:4379)
	at java.base/java.util.regex.Pattern$GroupHead.match(Pattern.java:4804)
	at java.base/java.util.regex.Pattern$GroupTail.match(Pattern.java:4863)
	at java.base/java.util.regex.Pattern$Curly.match0(Pattern.java:4392)
	at java.base/java.util.regex.Pattern$Curly.match(Pattern.java:4379)
	at java.base/java.util.regex.Pattern$GroupHead.match(Pattern.java:4804)
	at java.base/java.util.regex.Pattern$GroupTail.match(Pattern.java:4863)
	at java.base/java.util.regex.Pattern$Curly.match0(Pattern.java:4392)
	at java.base/java.util.regex.Pattern$Curly.match(Pattern.java:4379)
	at java.base/java.util.regex.Pattern$GroupHead.match(Pattern.java:4804)
	at java.base/java.util.regex.Pattern$GroupTail.match(Pattern.java:4863)
	at java.base/java.util.regex.Pattern$Curly.match0(Pattern.java:4392)
	at java.base/java.util.regex.Pattern$Curly.match(Pattern.java:4379)
	at java.base/java.util.regex.Pattern$GroupHead.match(Pattern.java:4804)
	at java.base/java.util.regex.Pattern$GroupTail.match(Pattern.java:4863)
	at java.base/java.util.regex.Pattern$Curly.match0(Pattern.java:4392)
	at java.base/java.util.regex.Pattern$Curly.match(Pattern.java:4379)
	at java.base/java.util.regex.Pattern$GroupHead.match(Pattern.java:4804)
	at java.base/java.util.regex.Pattern$GroupTail.match(Pattern.java:4863)
	at java.base/java.util.regex.Pattern$Curly.match0(Pattern.java:4392)
	at java.base/java.util.regex.Pattern$Curly.match(Pattern.java:4379)
	at java.base/java.util.regex.Pattern$GroupHead.match(Pattern.java:4804)
	at java.base/java.util.regex.Pattern$GroupTail.match(Pattern.java:4863)
	at java.base/java.util.regex.Pattern$Curly.match0(Pattern.java:4392)
	at java.base/java.util.regex.Pattern$Curly.match(Pattern.java:4379)
	at java.base/java.util.regex.Pattern$GroupHead.match(Pattern.java:4804)
...

Detail
When I'm mapping a plain text file using a custom reader, sometimes, the unmarshaller method thows a java.lang.StackOverflowError exception, probably because of the large amount of lines. It ussually happpens when the file has more than 100 lines but it has occurred on files with 60 or so.

Code

private static void setUp(Process process) {

        String defFile = getDefinitionFile(process);
        unmarshaller = new Unmarshaller<>(PedUtils.loadDefinition(defFile), new LineReader() {

           private SimpleLineReader simpleLineReader = new SimpleLineReader();

            public String readLine(Reader reader, Padder defaultPadder, String propertyDelimiter, int recordLength, String lineSeparator) {
                String line = simpleLineReader.readLine(reader, defaultPadder, propertyDelimiter, recordLength, lineSeparator);
                LOGGER.debug(process + "-Unmarshaller-Line read: " + line);
                if (line != null) {
                    if (line.contains("\r")) line = line.replace("\r", "");
                }
                return line;
            }
        });
    }

List<PedFile> file = unmarshaller.unmarshallToStream(PedUtils.openStream(getFileSource(process)))
                .collect(Collectors.toList());

file example:

1202101050045834582
200100458345829520513820201115202101050000250000000105
200200458345829520513800012021010500079344800008410695027022RON CENT 23A ZACAP  RON CENTENARIO 23 A ZACAPA 70 000010000000100000051900A000U 00005190           10000519000051900000051900005190000002500000051900005190000000000000000000000000021000000000000000
200200458345829520513800022021010500004579400008436532094088EST2 MATARROME CZA  EST2 DUE MATARROMERA TTO CZA  000010000000100000032390A001U 00003239           10000323900032390000035990003599000002500000032390003239000000000000000000000000021000000000000000
2002004583458295205138000320210105000220273000084203780615196FRI.YLLERA 5.5 25  PACK 6FRIZZ.YLLERA 5.5 VDJO 25000010000000100000015250A002U 00001525           10000152500015250000015250001525000002500000015250001525000000000000000000000000021000000000000000
200200458345829520513800042021010500003330300008423954260061RJA V.TONDONIA RVA  -RIOJA VIA TONDONIA RVA 75CL  000010000000100000030900A003U 00003090           10000309000030900000030900003090000002500000030900003090000000000000000000000000021000000000000000
...

[0]
https://github.com/ffissore/jrecordbind/blob/3.1.0/jrecordbind-test/src/test/java/org/fissore/jrecordbindtests/test/SimpleNotPaddedLineReaderUnmarshallTest.java

@ffissore
Copy link
Owner

ffissore commented Feb 7, 2021

Hi, and apologies for the late response. Do you mind sharing the XML Schema file as well?

@jzaratei
Copy link
Author

jzaratei commented Feb 8, 2021

Hi! I uploaded a gist with the xml definition and a file example

https://gist.github.com/jzaratei/f08cab2b15bdb04ae9b92fbec5591129

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants