SecureFile 和 DBFS

其他 2020-01-12 10:34:01 阅读次数: 0

Oracle Multimedia

这个特性在19c不再支持，参见说明。
不过多媒体文件仍建议存放在SecureFiles LOB中，参见此文。

SecureFiles初探

本实验参照Oracle Learning Library中的Using SecureFiles to Improve Performance, Maximize Storage, and Enhance Security。
关于SecureFiles，还有一篇很好的文章。
SecureFiles是BasicFiles的演进，11g开始支持。

SQL> show parameter db_securefile;

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_securefile                        string      PREFERRED


$ mkdir $ORACLE_HOME/wallet

编辑$ORACLE_HOME/network/admin/sqlnet.ora文件，加入以下，以指定wallet位置：

ENCRYPTION_WALLET_LOCATION= (SOURCE=(METHOD=FILE)(METHOD_DATA= (DIRECTORY=/opt/oracle/product/19c/dbhome_1/wallet)))

解压sf.zip到临时目录，本例为/tmp。
进入临时目录，登录到数据库，运行脚本obe_setup.sql，注意，由于我使用的是PDB，因此所有的connect语句均需修改，本例修改两处，为connect sys/Welcome1@orclpdb1 as sysdba和connect sf_demo/oracle@orclpdb1:

$ cd /tmp
$ ls -l
total 184
-rw-r--r--. 1 oracle oinstall   177 May 30  2007 check_encrypt.sql
-rw-r--r--. 1 oracle oinstall 37376 May 30  2007 david.sloan.doc
-rw-r--r--. 1 oracle oinstall   189 Mar  5  2007 dedup_compress.sql
-rw-r--r--. 1 oracle oinstall   177 May 30  2007 encrypt_lob.sql
-rw-r--r--. 1 oracle oinstall 64000 May 30  2007 karl.brimmer.doc
-rw-r--r--. 1 oracle oinstall  2554 May 30  2007 mig_bf_to_sf.sql
-rw-r--r--. 1 oracle oinstall 34304 May 30  2007 monica.petera.doc
-rw-r--r--. 1 oracle oinstall  8894 May 30  2007 obe_setup.sql
-rw-r--r--. 1 oracle oinstall   184 Apr 22  2009 read_data.sql
-rw-r--r--. 1 oracle oinstall   255 Mar  5  2007 reclaim_space.sql
-rw-r--r--. 1 oracle oinstall   118 May 30  2007 space_usage.sql
-rw-r--r--. 1 oracle oinstall   230 May 30  2007 write_data.sql
...
$ sqlplus /nolog
SQL> @obe_setup.sql

原脚本缺了给sf_demo用户赋表空间写的权限，补上：

SQL> alter user sf_demo quota unlimited on obe_tbs1;
User altered.

SQL> alter user sf_demo quota unlimited on obe_tbs2;
User altered.

另外，PDB中的wallet也需要打开，并创建master key，注意口令是oracle，与之前设定的一致：

SQL> show con_name;

CON_NAME
------------------------------
ORCLPDB1

SQL> ADMINISTER KEY MANAGEMENT SET KEYSTORE open identified by "oracle";

keystore altered.
SQL> administer key management create key identified by "oracle" with backup;

keystore altered.
SQL> administer key management use key 'AQQK0+b2fk/Pv7hk3BedhlEAAAAAAAAAAAAAAAAAAAAAAAAAAAAA' identified by "oracle" with backup;

keystore altered.
SQL> select * from v$encryption_wallet;

WRL_TYPE             WRL_PARA STATUS   WALLET_TYPE          WALLET_OR KEYSTORE FULLY_BAC     CON_ID
-------------------- -------- -------- -------------------- --------- -------- --------- ----------
FILE                          OPEN     PASSWORD             SINGLE    UNITED   NO                 3

然后，写数据，write_data.sql中的连接串也需改为PDB：

$ sqlplus /nolog
SQL> @write_data
Connected.
Begin inserting rows...
Row 1 inserted.
...
Row 30 inserted.

PL/SQL procedure successfully completed.

Begin inserting rows...
Row 1 inserted.
...
Row 30 inserted.

PL/SQL procedure successfully completed.

Begin inserting rows...
Row 1 inserted.
...
Row 30 inserted.

PL/SQL procedure successfully completed.

timing for: load_data
Elapsed: 00:00:00.30

总共90条记录，也就是3个doc文件，每个重复插入30次。
然后读取数据：

$ sqlplus /nolog
SQL> @read_data
SQL> @read_data
Connected.
SQL> set serveroutput on
SQL> set verify on
SQL> set term on
SQL> set lines 200
SQL>
SQL> pause Press [Enter] to continue...
Press [Enter] to continue...

SQL>
SQL> timing start read_data
SQL> exec read_lob;
The length is: 64000
The ID is: 1
The blob is read: ???        >  ??                       x          z       ????    w
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
The length is: 64000
...
The length is: 64000
The ID is: 30
The blob is read: ???        >  ??                       x          z       ????    w
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
The length is: 34304
The ID is: 1
The blob is read: ???        >  ??                       >          @       ????    =
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
...
The length is: 34304
The ID is: 30
The blob is read: ???        >  ??                       >          @       ????    =
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
...
The length is: 37376
The ID is: 30
The blob is read: ???        >  ??                       D          F       ????    C
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

PL/SQL procedure successfully completed.

SQL> timing stop
timing for: read_data
Elapsed: 00:00:00.11

确认空间使用，输入BF，表示老的Basic File格式：

$ sqlplus /nolog
SQL> @space_usage
Connected.
SQL>
SQL> exec check_space_&Enter_BF_or_SF_for_LOB_type
Enter value for enter_bf_or_sf_for_lob_type: BF
FS1 Blocks = 0 Bytes = 0
FS2 Blocks = 0 Bytes = 0
FS3 Blocks = 0 Bytes = 0
FS4 Blocks = 0 Bytes = 0
Full Blocks = 540 Bytes = 4423680
Unformatted Blocks = 7618 Bytes = 62406656
=============================================
Total Blocks = 540 ||  Total Bytes = 4423680

PL/SQL procedure successfully completed.

将BasicFile LOB 迁移到 SecureFiles：

SQL> @mig_bf_to_sf
SQL> -- run this script as SYS
SQL> connect sys/Welcome1@orclpdb1 as sysdba
Connected.
SQL>
SQL> set echo on;
SQL> set serveroutput on
SQL> set term on;
SQL> set lines 200
SQL>
SQL> set pause off
SQL>
SQL> /*== Check the current segment type of the LOB ==*/
SQL> /*==                                           ==*/
SQL> /*== Note that the segment subtype for         ==*/
SQL> /*== BasicFile LOB storage is ASSM             ==*/
SQL>
SQL> SELECT segment_name, segment_type, segment_subtype
  2  FROM dba_segments
  3  WHERE tablespace_name = 'OBE_TBS1'
  4  AND segment_type = 'LOBSEGMENT'
  5  /

SEGMENT_NAME                                                                                                                     SEGMENT_TYPE       SEGMENT_SU
-------------------------------------------------------------------------------------------------------------------------------- ------------------ ----------
SYS_LOB0000073784C00004$$                                                                                                        LOBSEGMENT         ASSM

SQL>
SQL> pause Press [Enter] to continue...
Press [Enter] to continue...

SQL>
SQL> set pause on
SQL>
SQL> /*== Create the interim table for online redefinition ==*/
SQL> /*==
SQL> /*== Note that the recommended practice is to enable  ==*/
SQL> /*== COMPRESSION and DEDUPLICATION at table createion ==*/
SQL> /*== time. Also, the ALTER TABLE SHRINK operation is  ==*/
SQL> /*== not yet supported for SecureFiles LOBs in        ==*/
SQL> /*== release 1 of Oracle Database 11g.                ==*/
SQL>
SQL> CREATE TABLE sf_demo.resumes_interim
  2  (id NUMBER, first_name VARCHAR2(15),
  3  last_name VARCHAR2 (40), resume BLOB)
  4  LOB (resume) STORE AS SECUREFILE
  5  (TABLESPACE obe_tbs2
  6   COMPRESS HIGH
  7   DEDUPLICATE)
  8  /

Table created.

SQL>
SQL> pause Press [Enter] to continue...
Press [Enter] to continue...

SQL>
SQL> /*== Perform the online redefinition ==*/
SQL>
SQL> DECLARE
  2   error_count PLS_INTEGER := 0;
  3  BEGIN
  4
  5  DBMS_REDEFINITION.START_REDEF_TABLE
  6  ('sf_demo', 'resumes', 'resumes_interim',
  7   'id id, first_name first_name, last_name last_name, resume resume',
  8   OPTIONS_FLAG => DBMS_REDEFINITION.CONS_USE_ROWID);
  9
 10  DBMS_REDEFINITION.COPY_TABLE_DEPENDENTS
 11  ('sf_demo', 'resumes', 'resumes_interim',
 12  1, true,true,true,false, error_count);
 13
 14  DBMS_OUTPUT.PUT_LINE('Errors := ' || TO_CHAR(error_count));
 15
 16  DBMS_REDEFINITION.FINISH_REDEF_TABLE
 17  ('sf_demo', 'resumes', 'resumes_interim');
 18
 19  END;
 20  /
Errors := 0

PL/SQL procedure successfully completed.

SQL>
SQL> pause Press [Enter] to continue...
Press [Enter] to continue...

SQL>
SQL> /*== Drop the interim table ==*/
SQL>
SQL> DROP TABLE sf_demo.resumes_interim
  2  /

Table dropped.

SQL>
SQL> pause Press [Enter] to continue...
Press [Enter] to continue...

SQL>
SQL> set pause off
SQL>
SQL> /*== Check the segment type of the migrated LOB ==*/
SQL> /*==                                            ==*/
SQL> /*== Note that the segment subtype for          ==*/
SQL> /*== SecureFiles LOB storage is SECUREFILE ==*/
SQL>
SQL> SELECT segment_name, segment_type, segment_subtype
  2  FROM dba_segments
  3  WHERE tablespace_name = 'OBE_TBS2'
  4  AND segment_type = 'LOBSEGMENT'
  5  /

SEGMENT_NAME                                                                                                                     SEGMENT_TYPE       SEGMENT_SU
-------------------------------------------------------------------------------------------------------------------------------- ------------------ ----------
SYS_LOB0000073803C00004$$                                                                                                        LOBSEGMENT         SECUREFILE

SQL> pause Press [Enter] to continue...
Press [Enter] to continue...

SQL>
SQL> set pause off
SQL>
SQL> /*== Check the DBA_LOBS data dictionary view for ==*/
SQL> /*== information on the SecureFiles COMPRESSION  ==*/
SQL> /*== and DEDUPLICATION settings                  ==*/
SQL>
SQL> SELECT column_name, segment_name,
  2         compression, deduplication, securefile
  3  FROM dba_lobs
  4  WHERE owner = 'SF_DEMO' and table_name = 'RESUMES'
  5  /

COLUMN_NAME
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SEGMENT_NAME                                                                                                                     COMPRE DEDUPLICATION   SEC
-------------------------------------------------------------------------------------------------------------------------------- ------ --------------- ---
RESUME
SYS_LOB0000073803C00004$$                                                                                                        HIGH   LOB             YES

迁移这一步使用了online redefinition，这是EBR的前身，有点意思。
迁移后，虽然segment的类型均为LOBSEGMENT，但子类型由ASSM变为了SECUREFILE。压缩和重复数据删除都启用了。
查看空间使用，输入SF表示SecureFile：

扫描二维码关注公众号，回复： 8536591 查看本文章 
SQL> @space_usage
SQL> connect sf_demo/oracle@orclpdb1
Connected.
SQL>
SQL> set serveroutput on
SQL> set pause off
SQL> set echo on
SQL>
SQL> exec check_space_&Enter_BF_or_SF_for_LOB_type
Enter value for enter_bf_or_sf_for_lob_type: SF
Segment Blocks = 8192 Bytes = 67108864
Used Blocks = 5 Bytes = 40960
Expired Blocks = 7962 Bytes = 65224704
Unexpired Blocks = 145 Bytes = 1187840
=============================================

PL/SQL procedure successfully completed.

注意到Used Block为5，而迁移前为540。

性能方面，SecureFile同样有优势，此处略。
最后是加密部分：

SQL> @encrypt_lob
SQL>
SQL> connect sys/Welcome1@orclpdb1 as sysdba
Connected.
SQL>
SQL> ALTER TABLE sf_demo.resumes
  2  MODIFY (resume ENCRYPT USING 'AES192')
  3  /

Table altered.

加密这部分，参考了此文档。
确认已加密：

SQL> @check_encrypt
SQL> connect sf_demo/oracle@orclpdb1
Connected.
SQL> set pause off
SQL> set echo on
SQL>
SQL> col TABLE_NAME format a18;
SQL> col COLUMN_NAME format a19;
SQL> col ENCRYPTION_ALG format a17;
SQL>
SQL> SELECT *
  2  FROM USER_ENCRYPTED_COLUMNS
  3  /

TABLE_NAME         COLUMN_NAME         ENCRYPTION_ALG    SAL INTEGRITY_AL
------------------ ------------------- ----------------- --- ------------
RESUMES            RESUME              AES 192 bits key  YES SHA-1

DBFS初探

本实验参照ORACLE-BASE上的文章：Oracle Database File System (DBFS) in Oracle Database 11g Release 2。
文章很全面，但我的环境是19.3，使用的是PDB，和它的11gR2有些区别。

创建表空间：

connect sys/Welcome1@orclpdb1 as sysdba
CREATE TABLESPACE dbfs_ts
  DATAFILE '/u01/app/oracle/oradata/DB11G/dbfs01.dbf'
  SIZE 1M AUTOEXTEND ON NEXT 1M;

创建文件系统:

cd $ORACLE_HOME/rdbms/admin
sqlplus dbfs_user/dbfs_user@orclpdb1
SQL> @dbfs_create_filesystem.sql dbfs_ts staging_area

安装FUSE，linux下的DBFS需要FUSE（Filesystem in Userspace）：

# yum install kernel-devel fuse fuse-libs

Mount文件系统：

# mkdir /mnt/dbfs
# chown oracle:oinstall /mnt/dbfs

配置动态链接库路径，参照文章12cR2部分，根据实际环境修改ORACLE_HOM环境变量：

# echo "/usr/local/lib" >> /etc/ld.so.conf.d/usr_local_lib.conf
# export ORACLE_HOME=/opt/oracle/product/19c/dbhome_1
# ln -s $ORACLE_HOME/lib/libclntsh.so.12.1 /usr/local/lib/libclntsh.so.12.1
# ln -s $ORACLE_HOME/lib/libnnz12.so /usr/local/lib/libnnz12.so
# ln -s /lib64/libfuse.so.2 /usr/local/lib/libfuse.so.2
# ln -s /lib64/libfuse.so.2 /usr/local/lib/libfuse.so
# ldconfig

修改/etc/fuse.conf文件，解除user_allow_other的注释：

# mount_max = 1000
user_allow_other

修改fusermount的权限：

# chmod +x /usr/bin/fusermount

修改/etc/abrt/abrt-action-save-package-data.conf部分，因我Linux 7环境下无此目录，略过。

重启服务器：

# shutdown -r now

在oracle用户的.bash_profile文件中添加以下：

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/usr/local/lib

mount文件系统：

$ dbfs_client dbfs_user@orclpdb1 /mnt/dbfs
Password: dbfs_user

这是个前台命令，通过df可确认文件系统已mount：

$ df /mnt/dbfs
Filesystem                1K-blocks  Used Available Use% Mounted on
dbfs-dbfs_user@orclpdb1:/       960   128       832  14% /mnt/dbfs

只有oracle用户可看到，root用户不行。
可以看到之前定义的staging_area文件系统已mount：

$ ls -al /mnt/dbfs/staging_area
total 0
drwxrwxrwx. 4 root   root     0 Dec 20 17:34 .
drwxr-xr-x. 3 root   root     0 Dec 21 12:21 ..
drwxr-xr-x. 7 root   root     0 Dec 20 08:13 .sfs

root用户可umount文件系统：

# fusermount -u /mnt/dbfs

dbfs_client命令示例：

# 查询文件系统
$ dbfs_client dbfs_user@orclpdb1 --command ls -a -l dbfs:/staging_area/
Password:
drwxr-xr-x              root            root               0    Dec 20 08:13    dbfs:/staging_area/.sfs
# 创建目录
$ dbfs_client dbfs_user@orclpdb1 --command mkdir dbfs:/staging_area/test_dir
Password:
# 查询确认
$ dbfs_client dbfs_user@orclpdb1 --command ls -a -l dbfs:/staging_area/
Password:
drwxr-xr-x            oracle        oinstall               0    Dec 21 13:10    dbfs:/staging_area/test_dir
drwxr-xr-x              root            root               0    Dec 20 08:13    dbfs:/staging_area/.sfs
# 拷贝文件
$ dbfs_client dbfs_user@orclpdb1 --command cp /tmp/flatxn.zip dbfs:/staging_area/test_dir/
Password:
/tmp/flatxn.zip -> dbfs:/staging_area/test_dir/flatxn.zip
# 删除文件
$ dbfs_client dbfs_user@orclpdb1 --command rm dbfs:/staging_area/test_dir/flatxn.zip
Password:
unlinking file dbfs:/staging_area/test_dir/flatxn.zip
# 再次拷贝
$ dbfs_client dbfs_user@orclpdb1 --command cp /tmp/flatxn.zip dbfs:/staging_area/test_dir/
Password:
/tmp/flatxn.zip -> dbfs:/staging_area/test_dir/flatxn.zip

DBFS还可以作为一种共享机制，只需在客户端安装dbfs_client即可，参见文章。
问题是，使用dbfs_client操作文件和使用操作系统下的命令操作有何区别？实际上，使用操作系统命令在服务器端操作是可以的，因为DBFS符合POSIX标准，但在客户端，你可以使用dbfs_client或scp。

$ cp /vagrant/dragonball.jpg /mnt/dbfs/staging_area/test_dir/

通过user_lobs，可知DBFS存放元数据的表为SFS$_FST_1。查询此表：
在这里插入图片描述
对于DBFS，可以使用PL/SQL API及系统视图得到许多信息，以下命令来自此文：

-- DBMS_DBFS_CONTENT_SFS
SELECT * FROM TABLE(dbms_dbfs_sfs.listTables);
SELECT * FROM TABLE(dbms_dbfs_sfs.listFilesystems);
SELECT * FROM TABLE(dbms_dbfs_sfs.listVolumes);
SELECT * FROM TABLE(dbms_dbfs_sfs.listSnapshots);
 
-- DBMS_DBFS_CONTENT
SELECT * FROM TABLE(dbms_dbfs_content.liststores);
SELECT * FROM TABLE(dbms_dbfs_content.listmounts);
SELECT * FROM TABLE(dbms_dbfs_content.listallcontent);
SELECT * FROM TABLE(dbms_dbfs_content.listallproperties);
SELECT * FROM TABLE(dbms_dbfs_content.list(path => '/', recurse => 1, store_name => 'staging_area'));

 -- Views 
 -- 除此命令外，其它命令均测试成功
SELECT UTL_RAW.cast_to_varchar2(filedata) AS filedata
FROM   dbfs_content
WHERE  pathtype = 'file';

SELECT * FROM dbfs_content_properties;

实际文件物理存放在表dbfs_content中：
在这里插入图片描述
最后，单个LOB最大为8TB，参见说明。

dingdingfish 博客专家
发布了342 篇原创文章 · 获赞 42 · 访问量 54万+
他的留言板 关注

猜你喜欢

转载自blog.csdn.net/stevensxiao/article/details/103634547
今日推荐
中国码农的“35岁魔咒”
蘭雅 CorelDRAW 插件 2024.5.1 国际劳动节版，免费下载
Arc Browser for Windows 1.0 正式 GA
90后程序员开发视频搬运软件、不到一年获利超 700 万，结局很刑！
《美国对全球网络空间安全与发展的威胁和破坏》报告发布
周排行
Java基础复习_day13_Collection集合
2018.11.16 c语言学习经验
且看Java内置四大核心函数式接口
小程序云开发中数据库的数据分段和显示图片
python的函数
Web-JS进阶
【干货】C++常用代码积累笔记大全
Spring的ioc操作 与 IOC底层原理
构建之法20191121-11 Scrum立会报告+燃尽图 07
Spring boot之Hello World访问404
每日归档
更多
2024-05-05(0)
2024-05-04(7)
2024-05-03(19)
2024-05-02(0)
2024-05-01(4)
2024-04-30(1)
2024-04-29(40)
2024-04-28(0)
2024-04-27(56)
2024-04-26(39)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022