Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add integration tests for volo #251

Open
PureWhiteWu opened this issue Oct 27, 2023 · 9 comments
Open

Add integration tests for volo #251

PureWhiteWu opened this issue Oct 27, 2023 · 9 comments
Labels
E-help-wanted Stuff where we want help.

Comments

@PureWhiteWu
Copy link
Member

Currently, we relies on the internal long-running stability testing services to ensure the stability of volo. This is not easily noticeable and cannot prevent errors from being merged in ci.

Thus, we need to add some integration tests for volo and run them in the github action.

I think we can just follow the guide in TRPL here.

I'll add some examples later when I have time.

@WSL0809
Copy link

WSL0809 commented Nov 16, 2023

I'd like to help and I noticed that you mentioned you would add some examples in the future. Please feel free to share them whenever you're ready; I'll be keeping an eye out for updates.

@Xuanwo
Copy link
Contributor

Xuanwo commented Dec 5, 2023

I will suggest adding a test to hive metastore, maybe the mostly used thrift rpc based services?

On my local test, calling hive metastore will meeting EOF errors:

+ERROR 1105 (HY000) at line 2: Internal. Code: 1001, Text = error on validating access
+thrift error: Transport(TransportError { kind: EndOfFile, message: "an unexpected end of file from server, rpc_info: RpcInfo { role: Client, caller: Some(Endpoint { service_name: \"\", address: None, faststr_tags: FastStrMap { inner: {} }, tags: TypeMap { inner: {} } }), callee: Some(Endpoint { service_name: \"hms\", address: Some(Ip(127.0.0.1:9083)), faststr_tags: FastStrMap { inner: {} }, tags: TypeMap { inner: {} } }), method: Some(\"get_database\"

@PureWhiteWu
Copy link
Member Author

@Xuanwo Hi, seems that that's because we defaults to use the TTHeader protocol which is a private protocol implemented by us, which hive doesn't support. For detailed information about this protocol, you can refer to: https://www.cloudwego.io/docs/kitex/reference/transport_protocol_ttheader/

This can be opted-out by calling make_codec when building the client.

Do you know which protocol hive uses? FramedBinary or BufferedBinary?

@Xuanwo
Copy link
Contributor

Xuanwo commented Dec 5, 2023

This can be opted-out by calling make_codec when building the client.

Great, let me try again.

(As a general thrift rpc framework, I suggest to disable those private protocol by default. Make it opt-in instead.)

Do you know which protocol hive uses? FramedBinary or BufferedBinary?

BufferedBinary

@PureWhiteWu
Copy link
Member Author

Does hive support framed transport? Framed transport is much more faster than simple buffered binary.

@Xuanwo
Copy link
Contributor

Xuanwo commented Dec 5, 2023

Does hive support framed transport? Framed transport is much more faster than simple buffered binary.

I'm not sure about that. AFAIK, Most hive users are using binary protocol.

@PureWhiteWu
Copy link
Member Author

(As a general thrift rpc framework, I suggest to disable those private protocol by default. Make it opt-in instead.)

Thanks for your suggestion! In fact, all CloudWeGo frameworks uses TTHeader transport by default (because this is the only way that can transmit metadata through services). The default protocol lacks this ability which is important for microservice development.

I'm not sure about that. AFAIK, Most hive users are using binary protocol.

Framed is a transport protocol, which adds a header indicates the length of payload. This is supported by most thrift frameworks, and I think you can try to use Framed first.

You can switch to use Framed transport by applying the following option when building client:

.make_codec(volo_thrift::codec::default::DefaultMakeCodec::framed())

@Xuanwo
Copy link
Contributor

Xuanwo commented Dec 5, 2023

Seems changing into framed can't resolve this issue. Hive does support framed transport, but it's not enabled by default.

+ERROR 1105 (HY000) at line 2: Internal. Code: 1001, Text = error on validating access
+thrift error: Transport(TransportError { kind: EndOfFile, message: "an unexpected end of file from server, rpc_info: RpcInfo { role: Client, caller: Some(Endpoint { service_name: \"\", address: None, faststr_tags: FastStrMap { inner: {} }, tags: TypeMap { inner: {} } }), callee: Some(Endpoint { service_name: \"hms\", address: Some(Ip(127.0.0.1:9083)), faststr_tags: FastStrMap { inner: {} }, tags: TypeMap { inner: {} } }), method: Some(\"get_database\"
+ERROR 1105 (HY000) at line 3: UnknownTable. Code: 1025, Text = error:

@PureWhiteWu
Copy link
Member Author

Oh, then could you please try the original buffered transport?
.make_codec(volo_thrift::codec::default::DefaultMakeCodec::buffered())

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
E-help-wanted Stuff where we want help.
Development

No branches or pull requests

3 participants