Stop letting employees secretly use ChatGPT: use a local network gateway to turn large models into "on-demand water
“Enterprises facing challenges in accessing large models encounter order issues. This article proposes a solution of deploying an open-source lightweight API gateway (llm-proxy-tk) within the internal network. This solution aims to achieve unified access to large models, issuance of cards per person, visibility into usage, and随时able disconnection of services, making them as controllable as company internal water and electricity services. The article points out three serious problems currently faced by small enterprises and teams using large models: first, keys are exposed openly in groups; second, lack of unified control leads to resource abuse; third, it is impossible to effectively monitor usage and costs.”