Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jq project: UTBot cannot generate tests, KLEE internal errors, Plugin cannot re-connect to the server #562

Open
tyuldashev opened this issue Dec 5, 2022 · 1 comment
Labels
bug Something isn't working wrong generation Inadequate or empty test suite generated

Comments

@tyuldashev
Copy link
Collaborator

Description
In most cases when tests are generated for jq project it results in error like 'function is too complex', 'syntax is not supported' or 'KLEE internal error'. Also after generation tests for one of the files plugin cannot re-connect to the server anymore

To Reproduce
Steps to reproduce the behavior:

  1. Clone and configure https://github.com/stedolan/jq project:
git clone https://github.com/stedolan/jq
cd jq
git submodule update --init

autoreconf -fi
cd build
../configure --with-oniguruma=builtin
  1. Add utbot_build.sh file with following content to project root:
cd build
make
  1. Configure UTBot through the wizard, so code is copied to the server

  2. Make utbot_build.sh executable on the server

  3. Re-configure UTBot once again if needed, so project targets appear in UTBot toolwindow

  4. Select jq target

  5. Generate and run tests for static jv type_error(jv bad, const char* msg) function in src/builtin.c file:

Expected behavior
Tests without errors generated and executed

Actual behavior
Generated tests contain following errors:

// Some tests for function 'type_error' were skipped, as execution of function leads KLEE to the internal error. See console log for more details.

Execution fails with error:

/home/utbot/remote/jq/tests/makefiles/src/../../../tests/src/builtin_dot_c_test.cpp:9:27: error: aggregate ‘OnigEncodingTypeST OnigEncodingUTF8’ has incomplete type and cannot be defined
    9 | struct OnigEncodingTypeST OnigEncodingUTF8;
      |                           ^~~~~~~~~~~~~~~~
/home/utbot/remote/jq/tests/makefiles/src/../../../tests/src/builtin_dot_c_test.cpp:10:1: error: ‘OnigSyntaxType’ does not name a type
   10 | OnigSyntaxType OnigSyntaxPerl_NG;
      | ^~~~~~~~~~~~~~
  1. Generate and run tests for file src/builtin.c:

Expected behavior
Tests are generated and run

Actual behavior
Execution fails with following errors:

/home/utbot/remote/jq/tests/makefiles/src/../../../tests/src/compile_dot_c_test.cpp: In member function ‘virtual void UTBot::error_block_join_test_6_Test::TestBody()’:
/home/utbot/remote/jq/tests/makefiles/src/../../../tests/src/compile_dot_c_test.cpp:4366:21: error: using typedef-name ‘UTBot::opcode’ after ‘enum’
 4366 |         .op = (enum opcode)(139711720),
      |                     ^~~~~~
In file included from /home/utbot/remote/jq/tests/makefiles/src/../../../tests/src/compile_dot_c_test.cpp:5:
/home/utbot/remote/jq/tests/makefiles/src/../../../tests/src/compile_dot_c_test.h:732:3: note: ‘UTBot::opcode’ has a previous declaration here
  732 | } opcode;
      |   ^~~~~~
  1. I didn't find exec steps to reproduce but most often hang happens on tests generation for whole file src/builtin.c or src/compile.c, after several executions generation hangs, after canceling when user tries to invoke generation error happens:
    image
    If user closes and opens project without restarting server then UTBot works again.
@tyuldashev tyuldashev added bug Something isn't working wrong generation Inadequate or empty test suite generated labels Dec 5, 2022
@ladisgin
Copy link
Member

ladisgin commented Feb 20, 2023

related issue
#587
#588

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working wrong generation Inadequate or empty test suite generated
Projects
Status: Todo
Development

No branches or pull requests

2 participants