STARK中的FRI代码解析

1. 引言

前序博客有：

• STARK入门知识
• STARKs and STARK VM: Proofs of Computational Integrity
• Fast Reed-Solomon Interactive Oracle Proofs of Proximity学习笔记

相关代码见：

• https://github.com/aszepieniec/stark-anatomy/blob/master/code（Python）

测试用例有：

$ pytest
=================================================================== test session starts ====================================================================
platform darwin -- Python 3.9.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /Users/lanyu/zyd/stark-anatomy/code
collected 21 items                                                                                                                                         

test_fast_stark.py F                                                                                                                                 [  4%]
test_fri.py .                                                                                                                                        [  9%]
test_ip.py F                                                                                                                                         [ 14%]
test_merkle.py .                                                                                                                                     [ 19%]
test_multivariate.py ..                                                                                                                              [ 28%]
test_ntt.py ......                                                                                                                                   [ 57%]
test_rescue_prime.py ..                                                                                                                              [ 66%]
test_rpsss.py ..                                                                                                                                     [ 76%]
test_stark.py F                                                                                                                                      [ 80%]
test_univariate.py ....                                                                                                                              [100%]

========================================================================= FAILURES =========================================================================
_____________________________________________________________________ test_fast_stark ______________________________________________________________________

    def test_fast_stark( ):
        field = Field.main()
        expansion_factor = 4
        num_colinearity_checks = 2
        security_level = 2
    
        rp = RescuePrime()
        output_element = field.sample(bytes(b'0xdeadbeef'))
    
        for trial in range(0, 20):
            input_element = output_element
            print("running trial with input:", input_element.value)
            output_element = rp.hash(input_element)
            num_cycles = rp.N+1
            state_width = rp.m
    
            stark = FastStark(field, expansion_factor, num_colinearity_checks, security_level, state_width, num_cycles)
            transition_zerofier, transition_zerofier_codeword, transition_zerofier_root = stark.preprocess()
    
            # prove honestly
            print("honest proof generation ...")
    
            # prove
            trace = rp.trace(input_element)
            air = rp.transition_constraints(stark.omicron)
            boundary = rp.boundary_constraints(output_element)
            proof = stark.prove(trace, air, boundary, transition_zerofier, transition_zerofier_codeword)
    
            # verify
            verdict = stark.verify(proof, air, boundary, transition_zerofier_root)
    
            assert(verdict == True), "valid stark proof fails to verify"
            print("success \\o/")
    
            print("verifying false claim ...")
            # verify false claim
            output_element_ = output_element + field.one()
            boundary_ = rp.boundary_constraints(output_element_)
            verdict = stark.verify(proof, air, boundary_, transition_zerofier_root)
    
            assert(verdict == False), "invalid stark proof verifies"
            print("proof rejected! \\o/")
    
            # prove with false witness
            print("attempting to prove with witness violating transition constraints (should not fail because using fast division) ...")
            cycle = 1 + (int(os.urandom(1)[0]) % len(trace)-1)
            register = int(os.urandom(1)[0]) % state_width
            error = field.sample(os.urandom(17))
    
            trace[cycle][register] = trace[cycle][register] + error
    
>           proof = stark.prove(trace, air, boundary, transition_zerofier, transition_zerofier_codeword)

test_fast_stark.py:60: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
fast_stark.py:97: in prove
    quotient = (trace_polynomials[s] - interpolant) / zerofier
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <univariate.Polynomial object at 0x10401b400>, other = <univariate.Polynomial object at 0x10401bc10>

    def __truediv__( self, other ):
        quo, rem = Polynomial.divide(self, other)
>       assert(rem.is_zero()), "cannot perform polynomial division because remainder is not zero"
E       AssertionError: cannot perform polynomial division because remainder is not zero

univariate.py:52: AssertionError
------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------
running trial with input: 228894434762048332457318
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 58193264164488245767062567324329845625
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 240442014346771117356623971907334212202
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 95249760241169493715201876097323148293
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 45460984185732302940594266735879849714
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 165587666758636920344584355360822203181
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 4978416156855947152397342202659038273
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 31067639684737653452690049877783213369
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 53584232234693103790669953564737900223
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 135431606538579019225905956689564915329
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 112559249994899591330577919758944336055
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 150140559072116678002193196297066253509
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 17074186389844667756333966236643273102
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 82836370955455571146008478781338270694
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 16254124710661698025820423055202445077
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 43769528645735998152380658131815329300
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 211513813789453019359177662714979719288
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 149981298211404449757454909679214149901
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
 ... but verification should fail :D
proof rejected! \o/
running trial with input: 196164108357361058832227489249577920500
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with witness violating transition constraints (should not fail because using fast division) ...
______________________________________________________________________ test_serialize ______________________________________________________________________

    def test_serialize( ):
        proof1 = ProofStream()
        proof1.push(1)
        proof1.push({
    
    1: '1'})
        proof1.push([1])
        proof1.push(2)
    
        serialized = proof1.serialize()
>       proof2 = ProofStream.deserialize(serialized)
E       TypeError: deserialize() missing 1 required positional argument: 'bb'

test_ip.py:11: TypeError
________________________________________________________________________ test_stark ________________________________________________________________________

    def test_stark( ):
        field = Field.main()
        expansion_factor = 4
        num_colinearity_checks = 2
        security_level = 2
    
        rp = RescuePrime()
        output_element = field.sample(bytes(b'0xdeadbeef'))
    
        for trial in range(0, 20):
            input_element = output_element
            print("running trial with input:", input_element.value)
            output_element = rp.hash(input_element)
            num_cycles = rp.N+1
            state_width = rp.m
    
            stark = Stark(field, expansion_factor, num_colinearity_checks, security_level, state_width, num_cycles)
    
            # prove honestly
            print("honest proof generation ...")
    
            # prove
            trace = rp.trace(input_element)
            air = rp.transition_constraints(stark.omicron)
            boundary = rp.boundary_constraints(output_element)
            proof = stark.prove(trace, air, boundary)
    
            # verify
            verdict = stark.verify(proof, air, boundary)
    
            assert(verdict == True), "valid stark proof fails to verify"
            print("success \\o/")
    
            print("verifying false claim ...")
            # verify false claim
            output_element_ = output_element + field.one()
            boundary_ = rp.boundary_constraints(output_element_)
            verdict = stark.verify(proof, air, boundary_)
    
            assert(verdict == False), "invalid stark proof verifies"
            print("proof rejected! \\o/")
    
        # verify with false witness
        print("attempting to prove with false witness (should fail) ...")
        cycle = int(os.urandom(1)[0]) % len(trace)
        register = int(os.urandom(1)[0]) % state_width
        error = field.sample(os.urandom(17))
    
        trace[cycle][register] = trace[cycle][register] + error
    
>       proof = stark.prove(trace, air, boundary) # should fail

test_stark.py:59: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
stark.py:111: in prove
    transition_quotients = [tp / self.transition_zerofier() for tp in transition_polynomials]
stark.py:111: in <listcomp>
    transition_quotients = [tp / self.transition_zerofier() for tp in transition_polynomials]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <univariate.Polynomial object at 0x104086130>, other = <univariate.Polynomial object at 0x10407d1c0>

    def __truediv__( self, other ):
        quo, rem = Polynomial.divide(self, other)
>       assert(rem.is_zero()), "cannot perform polynomial division because remainder is not zero"
E       AssertionError: cannot perform polynomial division because remainder is not zero

univariate.py:52: AssertionError
------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------
running trial with input: 228894434762048332457318
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 58193264164488245767062567324329845625
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 240442014346771117356623971907334212202
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 95249760241169493715201876097323148293
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 45460984185732302940594266735879849714
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 165587666758636920344584355360822203181
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 4978416156855947152397342202659038273
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 31067639684737653452690049877783213369
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 53584232234693103790669953564737900223
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 135431606538579019225905956689564915329
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 112559249994899591330577919758944336055
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 150140559072116678002193196297066253509
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 17074186389844667756333966236643273102
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 82836370955455571146008478781338270694
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 16254124710661698025820423055202445077
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 43769528645735998152380658131815329300
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 211513813789453019359177662714979719288
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 149981298211404449757454909679214149901
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 196164108357361058832227489249577920500
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
running trial with input: 141980557567619100445655384257862983893
honest proof generation ...
success \o/
verifying false claim ...
proof rejected! \o/
attempting to prove with false witness (should fail) ...
================================================================= short test summary info ==================================================================
FAILED test_fast_stark.py::test_fast_stark - AssertionError: cannot perform polynomial division because remainder is not zero
FAILED test_ip.py::test_serialize - TypeError: deserialize() missing 1 required positional argument: 'bb'
FAILED test_stark.py::test_stark - AssertionError: cannot perform polynomial division because remainder is not zero
======================================================== 3 failed, 18 passed in 2663.71s (0:44:23) =========================================================

2. FRI代码解析

$ pytest test_fri.py -s
================================================ test session starts =================================================
platform darwin -- Python 3.9.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /Users/lanyu/zyd/stark-anatomy/code
collected 1 item                                                                                                     

test_fri.py testing valid codeword ...

success! \o/
testing invalid codeword ...
last codeword does not correspond to polynomial of low enough degree
observed degree: 127
but should be: 31
success! \o/
.

================================================= 1 passed in 16.26s =================================================

以test_fri.py为例：

• 基于的域为p = 1 + 407 * ( 1 << 119 ) # 1 + 11 * 37 * 2^119
• 待证明的多项式degree $d=63$，待证明多项式为：$f(X)=c_0+c_{1}X+c_2{X}^2+\cdots +c_{63}{X}^{63}$，测试用例中的系数$[c_0,c_1,c_2,\cdots ,c_{63}]=[0,1,2,\cdots ,63]$。
• rate $\rho =1/4$，expansion factor为 $1/\rho =4$
• codeword length $N=(d+1)\ast \text{expansion\_factor}$，对应为evaluation set $S$的 size，evaluation set S = { ω 0 , ω 1 , ω 2 , ⋯   , ω 63 } S=\{\omega^0, \omega^1, \omega^2,\cdots,\omega^{63}\} S={ ω0,ω1,ω2,⋯,ω63}，其中$\omega$为$N$-th root of unity，$S$对应为代码中的domain，$\omega$对应为代码中的omega。
  codeword为 待证明多项式$f(X)$基于evaluation set $S$的 evaluation值集合，即 { f ( ω 0 ) , f ( ω 1 ) , f ( ω 2 ) , ⋯   , f ( ω 63 ) } \{f(\omega^0),f(\omega^1),f(\omega^2),\cdots,f(\omega^{63})\} { f(ω0),f(ω1),f(ω2),⋯,f(ω63)}。
• 根据FRI论文：
  
  可知，query complexity为$q=2\log N=16$，设置num_colinearity_tests=17。
  根据FRI论文可知，在query complexity（$q$）和 round complexity（$r$）之间做权衡，如$r=\log d/\log q$，$r$对应为代码中的num_rounds：

  	def num_rounds( self ):
          codeword_length = self.domain_length
          num_rounds = 0
          while codeword_length > self.expansion_factor and 4*self.num_colinearity_tests < codeword_length:
              codeword_length /= 2
              num_rounds += 1
          return num_rounds
  

基本流程为：

• 1）Commit阶段：

  • 1.1）在每一轮中，将该轮codeword构建为Merkle tree，将相应的root存入proof_stream中——发送给Verifier，将codeword存入codewords中——供后续提供query证明使用。

    	# compute and send Merkle root
        root = Merkle.commit(codeword)
        proof_stream.push(root)
        # collect codeword
        codewords += [codeword]
    

    而最后一轮，则直接将codeword发送给Verifier，同时也记录在codewords中。

    # send last codeword
    proof_stream.push(codeword)
    # collect last codeword too
    codewords = codewords + [codeword]
    

  • 1.2）在每一轮的Split-and-Fold过程中，challenge $\alpha$的计算方式为：

    # get challenge
    alpha = self.field.sample(proof_stream.prover_fiat_shamir())
    

  • 1.3）当将FRI与STARK一起结合使用时，由于STARK protocol中也定义了Reed-Solomon codeword，可能会存在point evaluation 相交的问题。
    因此，定义coset subgroup D = { g ⋅ ω i ∣ i ∈ Z } D = \lbrace g \cdot \omega^i \vert i \in \mathbb{Z}\rbrace D={ g⋅ωi∣i∈Z}，其中$g$为整个multiplicative group F \ { 0 } \mathbb{F} \backslash \lbrace 0\rbrace F\{ 0}的generator。而下一轮codeword所选择的evaluation domain 为the set of squares of $D$：
    D ⋆ = { d 2 ∣ d ∈ D } = { g 2 ⋅ ω 2 i ∣ i ∈ Z } D^\star = \lbrace d^2 \vert d \in D\rbrace = \lbrace g^2 \cdot \omega^{2i} \vert i \in \mathbb{Z}\rbrace D⋆={ d2∣d∈D}={ g2⋅ω2i∣i∈Z}
    其中，$g$对应为代码中的generator和offset。
    因此，在除第一轮之外的各轮中，codeword为对$f^{\star }(X)=2^{-1}\cdot \left((1+\alpha X^{-1})\cdot f(X)+(1-\alpha X^{-1})\cdot f(-X)\right)$ 基于coset subgroup D = { g ⋅ ω i ∣ i ∈ Z } D = \lbrace g \cdot \omega^i \vert i \in \mathbb{Z}\rbrace D={ g⋅ωi∣i∈Z}的evaluation值集合，即 { f ⋆ ( g ⋅ ω i ) } i = 0 , ⋯   , N / 2 \{f^\star(g\cdot \omega^i)\}_{i=0,\cdots,N/2} { f⋆(g⋅ωi)}i=0,⋯,N/2​：

    N = len(codeword)
    # split and fold
    codeword = [two.inverse() * ( (one + alpha / (offset * (omega^i)) ) * codeword[i] + (one - alpha / (offset * (omega^i)) ) * codeword[N//2 + i] ) for i in range(N//2)]