After studying the large model algorithm server for a long time, I found that the 4060Ti 16G is still cost-effective and can be assembled into a multi-card server.

1. Considering that the model’s graphics card uses 16G, it is enough.

You can directly run the 6B model without quantification. Generally it takes up about 12G.
Price of a single 4060Ti 16G ASUS

Relatively cheaper than Colorful, they are all considered first-tier graphics card manufacturers:
First-tier graphics card manufacturers: ASUS (ROG) MSI (Magic Dragon) Colorful (Vulcan) Zotac (Supreme) Gigabyte (Big Eagle)

https://www.zhihu.com/tardis/zm/art/428716561?source_id=1005

2. You can also use cheap x99+CPU+memory

Use the remaining money to buy a graphics card

3. Or you can also choose to use P100

https://www.topcpu.net/gpu-c/GeForce-RTX-4060-Ti-vs-Tesla-P100-PCIe-12-GB

https://www.bilibili.com/video/BV1Bh411w7HV/?spm_id_from=333.337.search-card.all.click&vd_source=4b290247452adda4e56d84b659b0c8a2

Now the price is almost the same as P40. P40 is also 800 yuan.