Build your own pip source on the intranet

1. Download the data package
2. Create index
3. Install nginx
4. Configure nginx
5. Display

1. Download the data package

Take Tsinghuayuan as an example

import requests
import bs4
import re
import wget
a = requests.get("https://pypi.tuna.tsinghua.edu.cn/simple").text
b = bs4.BeautifulSoup(a, "html.parser")
s = b.find_all("a")
count=0
with open("/Users/dongxiang/code/whl/pypi.txt", "w") as f:
   for i in range(len(s)):
      tree=requests.get("https://pypi.tuna.tsinghua.edu.cn/simple/"+s[i].text ).text
      tree_parser = bs4.BeautifulSoup(tree, "html.parser")
      real_name = tree_parser.find_all("a")
      for i in range(len(real_name)):
         ##print(str(real_name[i].text )+ "\n",str(count))
         count=count+1
         whlname=real_name[i].text
         ###此处只下载python3.10版本的whl
         if (re.findall(r'cp310', whlname) or re.findall(r'none-any', whlname) )and re.findall(r'.*\.whl', whlname) :
            #print(real_name[i].text+" URL：https://pypi.tuna.tsinghua.edu.cn/simple/"+real_name[i].attrs['href']+"\n")
            #f.write(real_name[i].text+" URL：https://pypi.tuna.tsinghua.edu.cn/simple/"+real_name[i].attrs['href']+"\n")
            wget.download("https://pypi.tuna.tsinghua.edu.cn/simple/s[i].text/"+real_name[i].attrs['href'],"/Users/dongxiang/code/whl"+real_name[i].text)
f.close()

2. Create index

You need to install python first, and it is best to create a virtual environment. Refer to my other article.

Install pip2pi locally

pip install pip2pi

Switch to the downloaded .whl folder on the command line and create an index (index.html is automatically generated)

dir2pi -S 【某文件夹】此处为/Users/dongxiang/code/whl

Then a simple folder appeared in the folder, and the content here is not that different from the sources of Alibaba, Tsinghua, and Douban.

Note that if you are in a Linux environment, the S must be lowercase.

3. Install nginx

Different system environments have different installation methods. You can find the details online.

Linux ：

yum install nginx

Step one: yum install nginx

After executing the installation command, wait for the installation to complete.
Step 2: cd /etc/nginx

After the installation is complete, the nginx directory
Step 3: service nginx start

If you can't access it, it may be a firewall problem.

添加 --permanent永久生效，没有此参数重启后失效 这里的6379为redis服务的端口，若为其他服务设置，需要对应端口
firewall-cmd --zone=public --add-port=6379/tcp --permanent 
重新载入
firewall-cmd --reload
查看所有打开的端口： 
firewall-cmd --zone=public --list-ports
防火墙的关闭： 
systemctl stop firewalld	//一般不用
防火墙的启动： 
systemctl start firewalld

View the installation directory

rpm -ql nginx

4. After running, you can directly enter the IP address to view. The default port is 80.

sudo service nginx start

5. Path to nginx default configuration file

/usr/local/nginx/nginx-1.13.7/conf

/etc/nginx/nginx.conf
Stop nginx: nginx.exe -s stop
Reload: nginx.exe -s reload
Exit: nginx.exe -s quit

Windows:
Just decompress the compressed package to the specified location.

4. Configure nginx

Modify the /etc/nginx/nginx.conf configuration file
Windows version
Insert image description here

Modify nginx.conf

server {
    
     
	listen 8888; 
	# server_name 10.10.2.33:8888; 
	server_name 127.0.0.1:8888; 
	charset utf-8; 
	location / {
    
     
	root C:/develop/Projects/pypip/whl/simple;
	autoindex on; 
	autoindex_exact_size off; #显示文件的大小
	autoindex_localtime on; #显示文件时间 
	#limit_rate_after 5m; #5分钟后下载速度限制为200k 
	#limit_rate 200k;
	} 
	access_log logs/pip.log; 
	}

Linux can refer to this modification.

5. Display

Insert image description here

After completion, you can delete the package you just downloaded directly, but the ones in the simple folder cannot be deleted. That is to say, the files in the upper layer of the simple folder can be deleted. The items in the red box can be deleted.
Insert image description here

If you need to add some packages temporarily.
Proceed as follows:
Now you need to add the package in whl2 to whl
Insert image description here

Put the price asking folder in whl2 into whl
Insert image description here

As shown in the picture
Insert image description here

Then modify the index.html in whl
and add the index.html in whl2 to the index.html in whl.
As shown in
Figure Figure 1
Insert image description here

Figure 2
Insert image description here
is best sorted alphabetically.

In addition, I wrote code to batch download Tsinghua sources and other sources to the local area. I won’t upload it here. If you have any questions, please email [email protected]