Repository navigation
Support embedding model environmental cost #169
Description
Activity
Hey @252afh :)
Although I understand the frustration, for now, our methodology is limited to text generation.
As this feature would require developing a new methodology and implementing it, I don't think that it is a priority right now, but it would clearly be a good middle/long-term goal!
Hey @adrienbanse !
Thanks for the response, I'd be interested in staying updated on any progress you make on this but agree that it's not a small feat!
Do you know of any other packages that serve the same purpose as ecologits but for embedding costs? I'm looking to create a paper-trail of accountability for our carbon usage so keen to get the entire picture captured :)
Ell
I'm not aware of any such software, sorry..
One comment though: I expect these costs to be much smaller than the text generation, but of course it depends on how much you request them.
We'll for sure let you know if such a feature is considered in Ecologits.
Reacted by Markus Tretzmüller
Description
I use ecologits for measuring
chat_completioncost through LiteLLM, but I can't see a way to get costs for embedding via ecologits.This is our current implementation in-progress i-dot-ai/i-dot-ai-utilities#8.
I want to use it to aggregate all of our carbon cost for all LLM calls. Currently I'm only able to get chat_completions, which feels like only half the pie.
I believe a lot of other people that use AI will need this for use in RAG systems especially, so it's worth embedding into ecologits for a single point-of-contact.