CaDiCal2019学习笔记（3）
内存预分配和垃圾回收
arena 竞技场，舞台
这单个文件arena.hpp arena.cpp collect.cpp 应当一起解读
1.在移动垃圾回收机制中生存下来的学习子句被移动到竞技场中，在移动子句时也控制子句的分配顺序。
This memory allocation arena provides fixed size pre-allocated memory for
the moving garbage collector 'copy_non_garbage_clauses' in 'collect.cpp'
to hold clauses which should survive garbage collection.
The advantage of using a pre-allocated arena is that the allocation order
of the clauses can be adapted in such a way that clauses watched by the
same literal are allocated consecutively.
采用预分配竞技场方法的好处是将子句按一定顺序组织分配空间，使得这些子句在观察体系中被考察的相同文字是连贯地。
This improves locality during
propagation and thus is more cache friendly. A similar technique is
implemented in MiniSAT and Glucose and gives substantial speed-up in
propagations per second even though it might even almost double peek
memory usage. Note that in MiniSAT this arena is actually required for
MiniSAT to be able to use 32 bit clauses references instead of 64 bit
pointers. This would restrict the maximum number of clauses and thus is
a restriction we do not want to use anymore.
New learned clauses are allocated in CaDiCaL outside of this arena and
moved to the arena during garbage collection.
译文：CaDiCaL中新的习得子句在这个舞台之外分配，并在垃圾收集期间移动到竞技场(在移动子句时也控制子句的分配顺序)。
The additional 'to' space
required for such a moving garbage collector is only allocated for those
clauses surviving garbage collection, which usually needs much less
memory than all clauses.
移动垃圾选择器会请求The additional 'to' space只给那部分在垃圾收集期间生存下来的子集分配空间。
The net effect is that in our implementation
the moving garbage collector using this arena only needs roughly 50% more
memory than allocating the clauses directly.
译文：我们的实施使用这个竞技场的移动垃圾收集器，最终的结果就是，只需要大约50%的额外内存资源，而不是直接分配子句。
Both implementations can be compared by varying the 'opts.arenatype' option (which also controls the allocation order of clauses during moving them).
译文：可以通过改变“opts.arenatype"选项来比较这两种实现。(在移动子句时也控制子句的分配顺序)。
The standard sequence of using the arena is as follows:
//重复流程：准备--拷贝、拷贝....-- 断言 --- 互换 ---没有在竞技场的被delete
//
// Arena arena;
// ...
// arena.prepare (bytes);
// q1 = arena.copy (p1, bytes1);
// ...
// qn = arena.copy (pn, bytesn);
// assert (bytes1 + ... + bytesn <= bytes);
// arena.swap ();
// ...
// if (!arena.contains (q)) delete q;
// ...
// arena.prepare (bytes);
// q1 = arena.copy (p1, bytes1);
// ...
// qn = arena.copy (pn, bytesn);
// assert (bytes1 + ... + bytesn <= bytes);
// arena.swap ();
// ...
//
// One has to be really careful with 'qi' references to arena memory.
2.竞技场类型声明：
struct Internal; //前置声明，竞技场类型Arena中包含Internal指针型数据成员
class Arena
{
Internal * internal;
struct { char * start, * top, * end; } from, to; //'to'登场了！
public:
Arena (Internal *); //带有一个参数的构造函数
~Arena ();
// Prepare 'to' space to hold that amount of memory. Precondition is that
// the 'to' space is empty. The following sequence of 'copy' operations
// can use as much memory in sum as pre-allocated here.
//
void prepare (size_t bytes);
// Does the memory pointed to by 'p' belong to this arena? More precisely
// to the 'from' space, since that is the only one remaining after 'swap'.
//
bool contains (void * p) const {
char * c = (char *) p;
return from.start <= c && c < from.top;
}
// Allocate that amount of memory in 'to' space. This assumes the 'to'
// space has been prepared to hold enough memory with 'prepare'. Then
// copy the memory pointed to by 'p' of size 'bytes'. Note that it does
// not matter whether 'p' is in 'from' or allocated outside of the arena.
//
char * copy (const char * p, size_t bytes) {
char * res = to.top;
to.top += bytes;
assert (to.top <= to.end);
memcpy (res, p, bytes);
return res;
}
// Completely delete 'from' space and then replace 'from' by 'to' (by
// pointer swapping). Everything previously allocated (in 'from') and not
// explicitly copied to 'to' with 'copy' becomes invalid.
//
void swap ();
};
3. collect.cpp文件中的函数解读
collect.cpp实际上是Internal类中涉及到其arena成员的相关操作的实现文件
1 #include "internal.hpp" 2 3 namespace CaDiCaL { 4 5 /*------------------------------------------------------------------------*/ 6 7 // Returns the positive number '1' ( > 0) if the given clause is root level 8 // satisfied or the negative number '-1' ( < 0) if it is not root level 9 // satisfied but contains a root level falsified literal. Otherwise, if it 10 // contains neither a satisfied nor falsified literal, then '0' is returned. 11 12 int Internal::clause_contains_fixed_literal (Clause * c) { 13 int satisfied = 0, falsified = 0; 14 for (const auto & lit : *c) { 15 const int tmp = fixed (lit); 16 if (tmp > 0) { 17 LOG (c, "root level satisfied literal %d in", lit); 18 satisfied++; 19 } 20 if (tmp < 0) { 21 LOG (c, "root level falsified literal %d in", lit); 22 falsified++; 23 } 24 } 25 if (satisfied) return 1; 26 else if (falsified) return -1; 27 else return 0; 28 } 29 30 // Assume that the clause is not root level satisfied but contains a literal 31 // set to false (root level falsified literal), so it can be shrunken. The 32 // clause data is not actually reallocated at this point to avoid dealing 33 // with issues of special policies for watching binary clauses or whether a 34 // clause is extended or not. Only its size field is adjusted accordingly 35 // after flushing out root level falsified literals. 36 37 void Internal::remove_falsified_literals (Clause * c) { 38 const const_literal_iterator end = c->end (); 39 const_literal_iterator i; 40 int num_non_false = 0; 41 for (i = c->begin (); num_non_false < 2 && i != end; i++) 42 if (fixed (*i) >= 0) num_non_false++; 43 if (num_non_false < 2) return; 44 if (proof) proof->flush_clause (c); 45 literal_iterator j = c->begin (); 46 for (i = j; i != end; i++) { 47 const int lit = *j++ = *i, tmp = fixed (lit); 48 assert (tmp <= 0); 49 if (tmp >= 0) continue; 50 LOG ("flushing %d", lit); 51 j--; 52 } 53 stats.collected += shrink_clause (c, j - c->begin ()); 54 } 55 56 // If there are new units (fixed variables) since the last garbage 57 // collection we go over all clauses, mark satisfied ones as garbage and 58 // flush falsified literals. Otherwise if no new units have been generated 59 // since the last garbage collection just skip this step. 60 61 void Internal::mark_satisfied_clauses_as_garbage () { 62 63 if (last.collect.fixed >= stats.all.fixed) return; 64 last.collect.fixed = stats.all.fixed; 65 66 LOG ("marking satisfied clauses and removing falsified literals"); 67 68 for (const auto & c : clauses) { 69 if (c->garbage) continue; 70 const int tmp = clause_contains_fixed_literal (c); 71 if (tmp > 0) mark_garbage (c); 72 else if (tmp < 0) remove_falsified_literals (c); 73 } 74 } 75 76 /*------------------------------------------------------------------------*/ 77 78 // Update occurrence lists before deleting garbage clauses in the context of 79 // preprocessing, e.g., during bounded variable elimination 'elim'. The 80 // result is the number of remaining clauses, which in this context means 81 // the number of non-garbage clauses. 82 83 size_t Internal::flush_occs (int lit) { 84 Occs & os = occs (lit); 85 const const_occs_iterator end = os.end (); 86 occs_iterator j = os.begin (); 87 const_occs_iterator i; 88 size_t res = 0; 89 Clause * c; 90 for (i = j; i != end; i++) { 91 c = *i; 92 if (c->collect ()) continue; 93 *j++ = c->moved ? c->copy : c; 94 assert (!c->redundant); 95 res++; 96 } 97 os.resize (j - os.begin ()); 98 shrink_occs (os); 99 return res; 100 } 101 102 // Update watch lists before deleting garbage clauses in the context of 103 // 'reduce' where we watch and no occurrence lists. We have to protect 104 // reason clauses not be collected and thus we have this additional check 105 // hidden in 'Clause.collect', which for the root level context of 106 // preprocessing is actually redundant. 107 108 inline void Internal::flush_watches (int lit, Watches & saved) { 109 assert (saved.empty ()); 110 Watches & ws = watches (lit); 111 const const_watch_iterator end = ws.end (); 112 watch_iterator j = ws.begin (); 113 const_watch_iterator i; 114 for (i = j; i != end; i++) { 115 Watch w = *i; 116 Clause * c = w.clause; 117 if (c->collect ()) continue; 118 if (c->moved) c = w.clause = c->copy; 119 w.size = c->size; 120 const int new_blit_pos = (c->literals[0] == lit); 121 assert (c->literals[!new_blit_pos] == lit); 122 w.blit = c->literals[new_blit_pos]; 123 if (w.binary ()) *j++ = w; 124 else saved.push_back (w); 125 } 126 ws.resize (j - ws.begin ()); 127 for (const auto & w : saved) ws.push_back (w); 128 saved.clear (); 129 shrink_vector (ws); 130 } 131 132 void Internal::flush_all_occs_and_watches () { 133 if (occs ()) 134 for (int idx = 1; idx <= max_var; idx++) 135 flush_occs (idx), flush_occs (-idx); 136 137 if (watches ()) { 138 Watches tmp; 139 for (int idx = 1; idx <= max_var; idx++) 140 flush_watches (idx, tmp), flush_watches (-idx, tmp); 141 } 142 } 143 144 /*------------------------------------------------------------------------*/ 145 146 // This is a simple garbage collector which does not move clauses. It needs 147 // less space than the arena based clause allocator, but is not as cache 148 // efficient, since the copying garbage collector can put clauses together 149 // which are likely accessed after each other. 150 151 void Internal::delete_garbage_clauses () { 152 153 flush_all_occs_and_watches (); 154 155 LOG ("deleting garbage clauses"); 156 long collected_bytes = 0, collected_clauses = 0; 157 const auto end = clauses.end (); 158 auto j = clauses.begin (), i = j; 159 while (i != end) { 160 Clause * c = *j++ = *i++; 161 if (!c->collect ()) continue; 162 collected_bytes += c->bytes (); 163 collected_clauses++; 164 delete_clause (c); 165 j--; 166 } 167 clauses.resize (j - clauses.begin ()); 168 shrink_vector (clauses); 169 170 PHASE ("collect", stats.collections, 171 "collected %ld bytes of %ld garbage clauses", 172 collected_bytes, collected_clauses); 173 } 174 175 /*------------------------------------------------------------------------*/ 176 177 // This is the start of the copying garbage collector using the arena. At 178 // the core is the following function, which copies a clause to the 'to' 179 // space of the arena. Be careful if this clause is a reason of an 180 // assignment. In that case update the reason reference. 181 // 182 void Internal::copy_clause (Clause * c) { 183 LOG (c, "moving"); 184 assert (!c->moved); 185 char * p = (char*) c, * q = arena.copy (p, c->bytes ()); 186 Clause * d = c->copy = (Clause *) q; 187 LOG ("copied clause[%p] to clause[%p]", c, d); 188 if (d->reason) { 189 assert (level > 0); 190 Var & v = var (d->literals[0]); 191 if (v.reason == c) v.reason = d; 192 else { 193 Var & u = var (d->literals[1]); 194 assert (u.reason == c); 195 u.reason = d; 196 } 197 } 198 c->moved = true; 199 } 200 201 // This is the moving garbage collector. 202 203 void Internal::copy_non_garbage_clauses () { 204 205 size_t collected_clauses = 0, collected_bytes = 0; 206 size_t moved_clauses = 0, moved_bytes = 0; 207 208 // First determine 'moved_bytes' and 'collected_bytes'. 209 // 210 for (const auto & c : clauses) 211 if (!c->collect ()) moved_bytes += c->bytes (), moved_clauses++; 212 else collected_bytes += c->bytes (), collected_clauses++; 213 214 PHASE ("collect", stats.collections, 215 "moving %ld bytes %.0f%% of %ld non garbage clauses", 216 (long) moved_bytes, 217 percent (moved_bytes, collected_bytes + moved_bytes), 218 (long) moved_clauses); 219 220 // Prepare 'to' space of size 'moved_bytes'. 221 // 222 arena.prepare (moved_bytes); 223 224 // Keep clauses in arena in the same order. 225 // 226 if (opts.arenacompact) 227 for (const auto & c : clauses) 228 if (!c->collect () && arena.contains (c)) 229 copy_clause (c); 230 231 if (opts.arenatype == 1 || !watches ()) { 232 233 // Localize according to current clause order. 234 235 // If the option 'opts.arenatype == 1' is set, then this means the 236 // solver uses the original order of clauses. If there are no watches, 237 // we can not use the watched based copying policies below. This 238 // happens if garbage collection is triggered during bounded variable 239 // elimination. 240 241 // Copy clauses according to the order of calling 'copy_clause', which 242 // in essence just gives a compacting garbage collector, since their 243 // relative order is kept, and actually already gives the largest 244 // benefit due to better cache locality. 245 246 for (const auto & c : clauses) 247 if (!c->moved && !c->collect ()) 248 copy_clause (c); 249 250 } else if (opts.arenatype == 2) { 251 252 // Localize according to (original) variable order. 253 254 // This is almost the version used by MiniSAT and descendants. 255 // Our version uses saved phases too. 256 257 for (int sign = -1; sign <= 1; sign += 2) 258 for (int idx = 1; idx <= max_var; idx++) 259 for (const auto & w : watches (sign * likely_phase (idx))) 260 if (!w.clause->moved && !w.clause->collect ()) 261 copy_clause (w.clause); 262 263 } else { 264 265 // Localize according to decision queue order. 266 267 // This is the default for search. It allocates clauses in the order of 268 // the decision queue and also uses saved phases. It seems faster than 269 // the MiniSAT version and thus we keep 'opts.arenatype == 3'. 270 271 assert (opts.arenatype == 3); 272 273 for (int sign = -1; sign <= 1; sign += 2) 274 for (int idx = queue.last; idx; idx = link (idx).prev) 275 for (const auto & w : watches (sign * likely_phase (idx))) 276 if (!w.clause->moved && !w.clause->collect ()) 277 copy_clause (w.clause); 278 } 279 280 // Do not forget to move clauses which are not watched, which happened in 281 // a rare situation, and now is only left as defensive code. 282 // 283 for (const auto & c : clauses) 284 if (!c->collect () && !c->moved) 285 copy_clause (c); 286 287 // Update watches or occurrence lists. 288 // 289 flush_all_occs_and_watches (); 290 291 // Replace and flush clause references in 'clauses'. 292 // 293 const auto end = clauses.end (); 294 auto j = clauses.begin (), i = j; 295 for (; i != end; i++) { 296 Clause * c = *i; 297 if (c->collect ()) delete_clause (c); 298 else assert (c->moved), *j++ = c->copy, deallocate_clause (c); 299 } 300 clauses.resize (j - clauses.begin ()); 301 if (clauses.size () < clauses.capacity ()/2) shrink_vector (clauses); 302 303 if (opts.arenasort) 304 rsort (clauses.begin (), clauses.end (), pointer_rank ()); 305 306 // Release 'from' space completely and then swap 'to' with 'from'. 307 // 308 arena.swap (); 309 310 PHASE ("collect", stats.collections, 311 "collected %ld bytes %.0f%% of %ld garbage clauses", 312 (long) collected_bytes, 313 percent (collected_bytes, collected_bytes + moved_bytes), 314 (long) collected_clauses); 315 } 316 317 /*------------------------------------------------------------------------*/ 318 319 // Maintaining clause statistics is complex and error prone but necessary 320 // for proper scheduling of garbage collection, particularly during bounded 321 // variable elimination. With this function we can check whether these 322 // statistics are updated correctly. 323 324 void Internal::check_clause_stats () { 325 #ifndef NDEBUG 326 long irredundant = 0, redundant = 0, total = 0, irrbytes = 0; 327 for (const auto & c : clauses) { 328 if (c->garbage) continue; 329 if (c->redundant) redundant++; else irredundant++; 330 if (!c->redundant) irrbytes += c->bytes (); 331 total++; 332 } 333 assert (stats.current.irredundant == irredundant); 334 assert (stats.current.redundant == redundant); 335 assert (stats.current.total == total); 336 assert (stats.irrbytes == irrbytes); 337 #endif 338 } 339 340 /*------------------------------------------------------------------------*/ 341 342 bool Internal::arenaing () { 343 return opts.arena && (stats.collections > 1); 344 } 345 346 void Internal::garbage_collection () { 347 if (unsat) return; 348 START (collect); 349 report ('G', 1); 350 stats.collections++; 351 mark_satisfied_clauses_as_garbage (); 352 if (arenaing ()) copy_non_garbage_clauses (); 353 else delete_garbage_clauses (); 354 check_clause_stats (); 355 check_var_stats (); 356 report ('C', 1); 357 STOP (collect); 358 } 359 360 } //end namespace CaDiCaL