Phase out very verbose element_count functions #95
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This could have been done better.
I figured out the secret to declaring backwards-compatibility functions that are no longer in the header file. This doesn't work:
This works:
So now you know.
For the lulz, I left the python scripts using the old one for now just to show that they work. Let me know if you want those all updated in one go, though I'd prefer to just rework the python scripts completely (I have new bindings that behave a lot better wrt errors).
Anyway, this PR removes
rwkv_get_state_buffer_element_count
andrwkv_get_logits_buffer_element_count
in favor ofrwkv_get_state_len
andrwkv_get_logits_len
respectively. It also adds some new functionsrwkv_get_n_vocab
,rwkv_get_n_embed
, andrwkv_get_n_layer
for getting those parameters of the model, for people who are interested, andrwkv_init_state
.Specifically,
rwkv_get_n_vocab
will assist people automatically picking which tokenizer to use (50277 -> BPE, 65535 -> World),rwkv_get_n_layer
will assist people in GPU offloading (figuring out how many layers are on/off GPU),rwkv_get_n_embed
will assist people with poking at the RWKV hidden state, since some people want to do that for e.g. embeddings.There is also a new
rwkv_init_state
which initializes a state to default / 0 tokens. Useful for people who can't / don't want to track how many times they have calledrwkv_eval
in order to pass null.All in all this family of functions seems like a very useful and needed addition, and I don't think there'd be any problems with its inclusion.