start porting latest tgi

[flozi00]

What does this PR do?

@tgaddair its just for you, tracking progress now, please do not merge at the moment

This PR also introduces FP8 Linear and fp8 kv cache by vllm

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Was this discussed/approved via a Github issue or the discord / slack channel? Please add a link
  to it if that's the case.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[flozi00]

Mistral + eetq tested and working

[flozi00]

llama tested too

[flozi00]

Benchmark vs Main branch:
{"input_tokens_per_second": 14643, "output_tokens_per_second": 218} -- main
{"input_tokens_per_second": 15003, "output_tokens_per_second": 236} -- This PR

[flozi00]

awq tested
Sharding tested

[tgaddair]

Nice! LGTM. Can land and then test it out further on main.