XRootD
XrdPfc::Info Class Reference

Status of cached file. Can be read from and written into a binary file. More...

#include <XrdPfcInfo.hh>

+ Collaboration diagram for XrdPfc::Info:

Classes

struct  AStat
 Access statistics. More...
 
struct  Status
 
union  Status.__unnamed126__
 
struct  Status.__unnamed126__.__unnamed128__
 
struct  Store
 

Public Member Functions

 Info (XrdSysTrace *trace, bool prefetchBuffer=false)
 Constructor. More...
 
 ~Info ()
 Destructor. More...
 
void CalcCksumMd5 (unsigned char *buff, char *digest)
 
uint32_t CalcCksumStore ()
 Get cksum, MD5 is for backward compatibility with V2 and V3. More...
 
uint32_t CalcCksumSyncedAndAStats ()
 
void CompactifyAccessRecords ()
 Compactify access records to the configured maximum. More...
 
int CountBlocksNotWrittenInRng (int firstIdx, int lastIdx) const
 Check download status in given block range. More...
 
void DowngradeCkSumState (CkSumCheck_e css_ref)
 
size_t GetAccessCnt () const
 Get number of accesses. More...
 
int GetBitvecSizeInBytes () const
 Get size of download-state bit-vector in bytes. More...
 
long long GetBufferSize () const
 Get prefetch buffer size. More...
 
CkSumCheck_e GetCkSumState () const
 
const char * GetCkSumStateAsText () const
 
time_t GetCreationTime () const
 Get file size. More...
 
long long GetExpectedDataFileSize () const
 Get expected data file size. More...
 
long long GetFileSize () const
 Get file size. More...
 
const AStatGetLastAccessStats () const
 Get latest access stats. More...
 
int GetLastDownloadedBlock () const
 Get number of the last downloaded block. More...
 
bool GetLatestDetachTime (time_t &t) const
 Get latest detach time. More...
 
int GetNBlocks () const
 Get number of blocks represented in download-state bit-vector. More...
 
int GetNDownloadedBlocks () const
 Get number of downloaded blocks. More...
 
long long GetNDownloadedBytes () const
 Get number of downloaded bytes. More...
 
time_t GetNoCkSumTime () const
 
time_t GetNoCkSumTimeForUVKeep () const
 
XrdSysTraceGetTrace () const
 
int GetVersion ()
 Get version. More...
 
bool HasNoCkSumTime () const
 
bool IsCkSumAny () const
 
bool IsCkSumBoth () const
 
bool IsCkSumCache () const
 
bool IsCkSumNet () const
 
bool IsComplete () const
 Get complete status. More...
 
bool Read (XrdOssDF *fp, const char *dname, const char *fname=0)
 Read content of cinfo file into this object. More...
 
const std::vector< AStat > & RefAStats () const
 
const StoreRefStoredData () const
 Get stored data. More...
 
void ResetAllAccessStats ()
 Reset IO Stats. More...
 
void ResetCkSumCache ()
 
void ResetCkSumNet ()
 
void ResetNoCkSumTime ()
 
void ResizeBits ()
 Reserve bit vectors for file_size / buffer_size bytes. More...
 
void SetAllBitsSynced ()
 Mark all blocks as synced to disk. More...
 
void SetBitPrefetch (int i)
 Mark block as obtained through prefetch. More...
 
void SetBitSynced (int i)
 Mark block as synced to disk. More...
 
void SetBitWritten (int i)
 Mark block as written to disk. More...
 
void SetBufferSizeFileSizeAndCreationTime (long long bs, long long fs)
 
void SetCkSumState (CkSumCheck_e css)
 
bool TestBitPrefetch (int i) const
 Test if block at the given index has been prefetched. More...
 
bool TestBitWritten (int i) const
 Test if block at the given index is written to disk. More...
 
void UpdateDownloadCompleteStatus ()
 Update complete status. More...
 
bool Write (XrdOssDF *fp, const char *dname, const char *fname=0)
 
void WriteIOStat (Stats &s)
 Write bytes missed, hits, and disk. More...
 
void WriteIOStatAttach ()
 Write open time in the last entry of access statistics. More...
 
void WriteIOStatDetach (Stats &s)
 Write close time together with bytes missed, hits, and disk. More...
 
void WriteIOStatSingle (long long bytes_disk)
 Write single open/close time for given bytes read from disk. More...
 
void WriteIOStatSingle (long long bytes_disk, time_t att, time_t dtc)
 Write open/close with given time and bytes read from disk. More...
 

Static Public Attributes

static const char * m_traceID = "CInfo"
 
static const int s_defaultVersion = 4
 
static const char * s_infoExtension = ".cinfo"
 
static const size_t s_infoExtensionLen = strlen(Info::s_infoExtension)
 
static size_t s_maxNumAccess = 20
 

Protected Attributes

std::vector< AStatm_astats
 access records More...
 
int m_bitvecSizeInBits
 cached More...
 
unsigned char * m_buff_prefetch
 prefetch statistics More...
 
unsigned char * m_buff_synced
 disk written state vector More...
 
unsigned char * m_buff_written
 download state vector More...
 
bool m_complete
 cached; if false, set to true when missingBlocks hit zero More...
 
bool m_hasPrefetchBuffer
 constains current prefetch score More...
 
int m_missingBlocks
 cached, updated in SetBitWritten() More...
 
Store m_store
 
XrdSysTracem_trace
 
int m_version
 

Detailed Description

Status of cached file. Can be read from and written into a binary file.

Definition at line 40 of file XrdPfcInfo.hh.


Class Documentation

◆ XrdPfc::Info::Status.__unnamed126__

union XrdPfc::Info::Status.__unnamed126__

Definition at line 44 of file XrdPfcInfo.hh.

+ Collaboration diagram for XrdPfc::Info::Status.__unnamed126__:
Class Members
__unnamed126__ __unnamed__
unsigned int _raw_

◆ XrdPfc::Info::Status.__unnamed126__.__unnamed128__

struct XrdPfc::Info::Status.__unnamed126__.__unnamed128__

Definition at line 45 of file XrdPfcInfo.hh.

+ Collaboration diagram for XrdPfc::Info::Status.__unnamed126__.__unnamed128__:
Class Members
int _free_bits_: 29
int f_cksum_check: 3 as in enum CkSumCheck_e

Constructor & Destructor Documentation

◆ Info()

Info::Info ( XrdSysTrace trace,
bool  prefetchBuffer = false 
)

Constructor.

Definition at line 123 of file XrdPfcInfo.cc.

123  :
124  m_trace(trace),
126  m_version(0),
128  m_missingBlocks(0),
129  m_complete(false),
130  m_hasPrefetchBuffer(prefetchBuffer),
131  m_cksCalcMd5(0)
132 {}
unsigned char * m_buff_prefetch
prefetch statistics
Definition: XrdPfcInfo.hh:322
unsigned char * m_buff_written
download state vector
Definition: XrdPfcInfo.hh:321
unsigned char * m_buff_synced
disk written state vector
Definition: XrdPfcInfo.hh:320
bool m_complete
cached; if false, set to true when missingBlocks hit zero
Definition: XrdPfcInfo.hh:328
int m_missingBlocks
cached, updated in SetBitWritten()
Definition: XrdPfcInfo.hh:327
int m_bitvecSizeInBits
cached
Definition: XrdPfcInfo.hh:326
XrdSysTrace * m_trace
Definition: XrdPfcInfo.hh:317
bool m_hasPrefetchBuffer
constains current prefetch score
Definition: XrdPfcInfo.hh:329

◆ ~Info()

Info::~Info ( )

Destructor.

Definition at line 134 of file XrdPfcInfo.cc.

135 {
136  if (m_buff_synced) free(m_buff_synced);
137  if (m_buff_written) free(m_buff_written);
138  if (m_buff_prefetch) free(m_buff_prefetch);
139  delete m_cksCalcMd5;
140 }

References m_buff_prefetch, m_buff_synced, and m_buff_written.

Member Function Documentation

◆ CalcCksumMd5()

void Info::CalcCksumMd5 ( unsigned char *  buff,
char *  digest 
)

Definition at line 238 of file XrdPfcInfo.cc.

239 {
240  if (m_cksCalcMd5)
241  m_cksCalcMd5->Init();
242  else
243  m_cksCalcMd5 = new XrdCksCalcmd5();
244 
245  m_cksCalcMd5->Update((const char*)buff, GetBitvecSizeInBytes());
246  memcpy(digest, m_cksCalcMd5->Final(), 16);
247 }
virtual char * Final()=0
virtual void Update(const char *Buff, int BLen)=0
virtual void Init()=0
int GetBitvecSizeInBytes() const
Get size of download-state bit-vector in bytes.
Definition: XrdPfcInfo.hh:429

References XrdCksCalc::Final(), GetBitvecSizeInBytes(), XrdCksCalc::Init(), and XrdCksCalc::Update().

+ Here is the call graph for this function:

◆ CalcCksumStore()

uint32_t Info::CalcCksumStore ( )

Get cksum, MD5 is for backward compatibility with V2 and V3.

Definition at line 227 of file XrdPfcInfo.cc.

228 {
229  return crc32c(0, &m_store, sizeof(Store));
230 }
uint32_t crc32c(uint32_t crc, void const *buf, size_t len)

References crc32c(), and m_store.

Referenced by Read(), and Write().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ CalcCksumSyncedAndAStats()

uint32_t Info::CalcCksumSyncedAndAStats ( )

Definition at line 232 of file XrdPfcInfo.cc.

233 {
234  uint32_t cks = crc32c(0, m_buff_synced, GetBitvecSizeInBytes());
235  return crc32c(cks, m_astats.data(), m_astats.size() * sizeof(AStat));
236 }
std::vector< AStat > m_astats
access records
Definition: XrdPfcInfo.hh:323

References crc32c(), GetBitvecSizeInBytes(), m_astats, and m_buff_synced.

Referenced by Read(), and Write().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ CompactifyAccessRecords()

void Info::CompactifyAccessRecords ( )

Compactify access records to the configured maximum.

Definition at line 379 of file XrdPfcInfo.cc.

380 {
381  time_t now = time(0);
382 
383  std::vector<AStat> &v = m_astats;
384 
385  for (int i = 0; i < (int) v.size() - 1; ++i)
386  {
387  if (v[i].DetachTime == 0)
388  v[i].DetachTime = std::min(v[i].AttachTime + v[i].Duration / v[i].NumIos, v[i+1].AttachTime);
389  }
390 
391  while (v.size() > s_maxNumAccess)
392  {
393  double min_s = 1e10;
394  int min_i = -1;
395 
396  int M = (int) v.size() - 2;
397  for (int i = 0; i < M; ++i)
398  {
399  AStat &a = v[i], &b = v[i + 1];
400 
401  time_t t = std::max((time_t) 1, (now - b.AttachTime) / 2 + (now - a.DetachTime) / 2);
402  double s = (double) (b.AttachTime - a.DetachTime) / t;
403 
404  if (s < min_s)
405  {
406  min_s = s;
407  min_i = i;
408  }
409  }
410  assert(min_i != -1);
411 
412  v[min_i].MergeWith(v[min_i + 1]);
413 
414  v.erase(v.begin() + (min_i + 1));
415  }
416 }
static size_t s_maxNumAccess
Definition: XrdPfcInfo.hh:311

References XrdPfc::Info::AStat::DetachTime, m_astats, and s_maxNumAccess.

Referenced by Write().

+ Here is the caller graph for this function:

◆ CountBlocksNotWrittenInRng()

int XrdPfc::Info::CountBlocksNotWrittenInRng ( int  firstIdx,
int  lastIdx 
) const
inline

Check download status in given block range.

Definition at line 452 of file XrdPfcInfo.hh.

453 {
454  // TODO rewrite to use full byte comparisons outside of edges ?
455  // Also, it seems to be always called with firstIdx = 0, lastIdx = m_bitvecSizeInBits.
456  int cnt = 0;
457  for (int i = firstIdx; i < lastIdx; ++i)
458  if (! TestBitWritten(i)) ++cnt;
459 
460  return cnt;
461 }
bool TestBitWritten(int i) const
Test if block at the given index is written to disk.
Definition: XrdPfcInfo.hh:343

References TestBitWritten().

Referenced by UpdateDownloadCompleteStatus().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ DowngradeCkSumState()

void XrdPfc::Info::DowngradeCkSumState ( CkSumCheck_e  css_ref)
inline

Definition at line 295 of file XrdPfcInfo.hh.

295 { m_store.m_status.f_cksum_check &= css_ref; }
Status m_status
status information
Definition: XrdPfcInfo.hh:83

References XrdPfc::Info::Store::m_status, and m_store.

◆ GetAccessCnt()

size_t XrdPfc::Info::GetAccessCnt ( ) const
inline

Get number of accesses.

Definition at line 261 of file XrdPfcInfo.hh.

261 { return m_store.m_accessCnt; }
size_t m_accessCnt
total access count for the file
Definition: XrdPfcInfo.hh:82

References XrdPfc::Info::Store::m_accessCnt, and m_store.

Referenced by XrdPfc::File::GetAccessCnt().

+ Here is the caller graph for this function:

◆ GetBitvecSizeInBytes()

int XrdPfc::Info::GetBitvecSizeInBytes ( ) const
inline

Get size of download-state bit-vector in bytes.

Definition at line 429 of file XrdPfcInfo.hh.

430 {
431  if (m_bitvecSizeInBits)
432  return ((m_bitvecSizeInBits - 1)/8 + 1);
433  else
434  return 0;
435 }

References m_bitvecSizeInBits.

Referenced by CalcCksumMd5(), CalcCksumSyncedAndAStats(), Read(), ResizeBits(), SetAllBitsSynced(), SetBitPrefetch(), SetBitSynced(), SetBitWritten(), TestBitPrefetch(), TestBitWritten(), and Write().

+ Here is the caller graph for this function:

◆ GetBufferSize()

long long XrdPfc::Info::GetBufferSize ( ) const
inline

Get prefetch buffer size.

Definition at line 469 of file XrdPfcInfo.hh.

470 {
471  return m_store.m_buffer_size;
472 }
long long m_buffer_size
buffer / block size
Definition: XrdPfcInfo.hh:78

References XrdPfc::Info::Store::m_buffer_size, and m_store.

Referenced by XrdPfc::File::GetBlockSize().

+ Here is the caller graph for this function:

◆ GetCkSumState()

CkSumCheck_e XrdPfc::Info::GetCkSumState ( ) const
inline

Definition at line 286 of file XrdPfcInfo.hh.

286 { return (CkSumCheck_e) m_store.m_status.f_cksum_check; }

References XrdPfc::Info::Store::m_status, and m_store.

◆ GetCkSumStateAsText()

const char * Info::GetCkSumStateAsText ( ) const

Definition at line 249 of file XrdPfcInfo.cc.

250 {
251  switch (m_store.m_status.f_cksum_check) {
252  case CSChk_None : return "none";
253  case CSChk_Cache : return "cache";
254  case CSChk_Net : return "net";
255  case CSChk_Both : return "both";
256  default : return "unknown";
257  }
258 }
@ CSChk_Both
Definition: XrdPfcTypes.hh:27
@ CSChk_Net
Definition: XrdPfcTypes.hh:27
@ CSChk_Cache
Definition: XrdPfcTypes.hh:27
@ CSChk_None
Definition: XrdPfcTypes.hh:27

References XrdPfc::CSChk_Both, XrdPfc::CSChk_Cache, XrdPfc::CSChk_Net, XrdPfc::CSChk_None, XrdPfc::Info::Store::m_status, and m_store.

◆ GetCreationTime()

time_t XrdPfc::Info::GetCreationTime ( ) const
inline

Get file size.

Definition at line 277 of file XrdPfcInfo.hh.

277 { return m_store.m_creationTime; }
time_t m_creationTime
time the info file was created
Definition: XrdPfcInfo.hh:80

References XrdPfc::Info::Store::m_creationTime, and m_store.

◆ GetExpectedDataFileSize()

long long XrdPfc::Info::GetExpectedDataFileSize ( ) const
inline

Get expected data file size.

Definition at line 420 of file XrdPfcInfo.hh.

421 {
422  int last_block = GetLastDownloadedBlock();
423  if (last_block == m_bitvecSizeInBits - 1)
424  return m_store.m_file_size;
425  else
426  return (last_block + 1) * m_store.m_buffer_size;
427 }
int GetLastDownloadedBlock() const
Get number of the last downloaded block.
Definition: XrdPfcInfo.hh:412
long long m_file_size
size of file in bytes
Definition: XrdPfcInfo.hh:79

References GetLastDownloadedBlock(), m_bitvecSizeInBits, XrdPfc::Info::Store::m_buffer_size, XrdPfc::Info::Store::m_file_size, and m_store.

+ Here is the call graph for this function:

◆ GetFileSize()

long long XrdPfc::Info::GetFileSize ( ) const
inline

Get file size.

Definition at line 442 of file XrdPfcInfo.hh.

443 {
444  return m_store.m_file_size;
445 }

References XrdPfc::Info::Store::m_file_size, and m_store.

Referenced by XrdPfc::Cache::DetermineFullFileSize(), and XrdPfc::Cache::LocalFilePath().

+ Here is the caller graph for this function:

◆ GetLastAccessStats()

const Info::AStat * Info::GetLastAccessStats ( ) const

Get latest access stats.

Definition at line 489 of file XrdPfcInfo.cc.

490 {
491  return m_astats.empty() ? 0 : & m_astats.back();
492 }

References m_astats.

Referenced by XrdPfc::File::GetLastAccessStats().

+ Here is the caller graph for this function:

◆ GetLastDownloadedBlock()

int XrdPfc::Info::GetLastDownloadedBlock ( ) const
inline

Get number of the last downloaded block.

Definition at line 412 of file XrdPfcInfo.hh.

413 {
414  for (int i = m_bitvecSizeInBits - 1; i >= 0; --i)
415  if (TestBitWritten(i)) return i;
416 
417  return -1;
418 }

References m_bitvecSizeInBits, and TestBitWritten().

Referenced by GetExpectedDataFileSize().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ GetLatestDetachTime()

bool Info::GetLatestDetachTime ( time_t &  t) const

Get latest detach time.

Definition at line 470 of file XrdPfcInfo.cc.

471 {
472  if (m_astats.empty())
473  {
475  }
476  else
477  {
478  const AStat& ls = m_astats.back();
479 
480  if (ls.DetachTime == 0)
481  t = ls.AttachTime + ls.Duration;
482  else
483  t = ls.DetachTime;
484  }
485 
486  return t != 0;
487 }

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::AStat::Duration, m_astats, XrdPfc::Info::Store::m_creationTime, and m_store.

Referenced by XrdPfc::Cache::ExecuteCommandUrl().

+ Here is the caller graph for this function:

◆ GetNBlocks()

int XrdPfc::Info::GetNBlocks ( ) const
inline

Get number of blocks represented in download-state bit-vector.

Definition at line 437 of file XrdPfcInfo.hh.

438 {
439  return m_bitvecSizeInBits;
440 }

References m_bitvecSizeInBits.

Referenced by XrdPfc::File::GetNBlocks().

+ Here is the caller graph for this function:

◆ GetNDownloadedBlocks()

int XrdPfc::Info::GetNDownloadedBlocks ( ) const
inline

Get number of downloaded blocks.

Definition at line 398 of file XrdPfcInfo.hh.

399 {
400  int cntd = 0;
401  for (int i = 0; i < m_bitvecSizeInBits; ++i)
402  if (TestBitWritten(i)) cntd++;
403 
404  return cntd;
405 }

References m_bitvecSizeInBits, and TestBitWritten().

Referenced by XrdPfc::File::GetNDownloadedBlocks(), and GetNDownloadedBytes().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ GetNDownloadedBytes()

long long XrdPfc::Info::GetNDownloadedBytes ( ) const
inline

Get number of downloaded bytes.

Definition at line 407 of file XrdPfcInfo.hh.

408 {
410 }
int GetNDownloadedBlocks() const
Get number of downloaded blocks.
Definition: XrdPfcInfo.hh:398

References GetNDownloadedBlocks(), XrdPfc::Info::Store::m_buffer_size, and m_store.

+ Here is the call graph for this function:

◆ GetNoCkSumTime()

time_t XrdPfc::Info::GetNoCkSumTime ( ) const
inline

Definition at line 300 of file XrdPfcInfo.hh.

300 { return m_store.m_noCkSumTime; }
time_t m_noCkSumTime
time when first non-cksummed block was detected
Definition: XrdPfcInfo.hh:81

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

◆ GetNoCkSumTimeForUVKeep()

time_t XrdPfc::Info::GetNoCkSumTimeForUVKeep ( ) const
inline

◆ GetTrace()

XrdSysTrace* XrdPfc::Info::GetTrace ( ) const
inline

Definition at line 314 of file XrdPfcInfo.hh.

314 {return m_trace; }

References m_trace.

◆ GetVersion()

int XrdPfc::Info::GetVersion ( )
inline

Get version.

Definition at line 266 of file XrdPfcInfo.hh.

266 { return m_version; }

References m_version.

◆ HasNoCkSumTime()

bool XrdPfc::Info::HasNoCkSumTime ( ) const
inline

Definition at line 299 of file XrdPfcInfo.hh.

299 { return m_store.m_noCkSumTime != 0; }

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

Referenced by ResetCkSumCache(), and ResetCkSumNet().

+ Here is the caller graph for this function:

◆ IsCkSumAny()

bool XrdPfc::Info::IsCkSumAny ( ) const
inline

Definition at line 291 of file XrdPfcInfo.hh.

291 { return m_store.m_status.f_cksum_check & CSChk_Both; }

References XrdPfc::CSChk_Both, XrdPfc::Info::Store::m_status, and m_store.

◆ IsCkSumBoth()

bool XrdPfc::Info::IsCkSumBoth ( ) const
inline

Definition at line 292 of file XrdPfcInfo.hh.

292 { return (m_store.m_status.f_cksum_check & CSChk_Both) == CSChk_Both; }

References XrdPfc::CSChk_Both, XrdPfc::Info::Store::m_status, and m_store.

◆ IsCkSumCache()

bool XrdPfc::Info::IsCkSumCache ( ) const
inline

Definition at line 289 of file XrdPfcInfo.hh.

289 { return m_store.m_status.f_cksum_check & CSChk_Cache; }

References XrdPfc::CSChk_Cache, XrdPfc::Info::Store::m_status, and m_store.

Referenced by ResetCkSumCache(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ IsCkSumNet()

bool XrdPfc::Info::IsCkSumNet ( ) const
inline

Definition at line 290 of file XrdPfcInfo.hh.

290 { return m_store.m_status.f_cksum_check & CSChk_Net; }

References XrdPfc::CSChk_Net, XrdPfc::Info::Store::m_status, and m_store.

Referenced by ResetCkSumNet(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ IsComplete()

bool XrdPfc::Info::IsComplete ( ) const
inline

Get complete status.

Definition at line 447 of file XrdPfcInfo.hh.

448 {
449  return m_complete;
450 }

References m_complete.

Referenced by XrdPfc::Cache::LocalFilePath(), XrdPfc::File::Read(), XrdPfc::File::ReadV(), XrdPfc::File::Sync(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ Read()

bool Info::Read ( XrdOssDF fp,
const char *  dname,
const char *  fname = 0 
)

Read content of cinfo file into this object.

Parameters
fpfile handle
dnamedirectory name for trace output
fnameoptional file name for trace output (can be included in dname)
Returns
true on success

Definition at line 294 of file XrdPfcInfo.cc.

295 {
296  // Does not need lock, called only in File::Open before File::Run() starts.
297  // XXXX Wait, how about Purge, and LocalFilePath, Stat?
298 
299  TraceHeader trace_pfx("Read()", dname, fname);
300 
301  FpHelper r(fp, 0, m_trace, m_traceID, trace_pfx);
302 
303  if (r.Read(m_version)) return false;
304 
306  {
307  if (m_version == 2)
308  {
309  return ReadV2(fp, r.f_off, dname, fname);
310  }
311  else if (m_version == 3)
312  {
313  return ReadV3(fp, r.f_off, dname, fname);
314  }
315  else
316  {
317  TRACE(Warning, trace_pfx << "File version " << m_version << " not supported.");
318  return false;
319  }
320  }
321 
322  uint32_t cksum;
323 
324  if (r.Read(m_store) || r.Read(cksum)) return false;
325 
326  if (cksum != CalcCksumStore())
327  {
328  TRACE(Error, trace_pfx << "Checksum Store mismatch.");
329  return false;
330  }
331 
332  ResizeBits();
333  m_astats.resize(m_store.m_astatSize);
334 
335  if (r.ReadRaw(m_buff_synced, GetBitvecSizeInBytes()) ||
336  r.ReadRaw(m_astats.data(), m_store.m_astatSize * sizeof(AStat)) ||
337  r.Read(cksum))
338  {
339  return false;
340  }
341 
342  if (cksum != CalcCksumSyncedAndAStats())
343  {
344  TRACE(Error, trace_pfx << "Checksum Synced or AStats mismatch.");
345  return false;
346  }
347 
349 
351 
352  return true;
353 }
@ Warning
#define TRACE(act, x)
Definition: XrdTrace.hh:63
void UpdateDownloadCompleteStatus()
Update complete status.
Definition: XrdPfcInfo.hh:463
uint32_t CalcCksumStore()
Get cksum, MD5 is for backward compatibility with V2 and V3.
Definition: XrdPfcInfo.cc:227
void ResizeBits()
Reserve bit vectors for file_size / buffer_size bytes.
Definition: XrdPfcInfo.cc:172
static const int s_defaultVersion
Definition: XrdPfcInfo.hh:312
uint32_t CalcCksumSyncedAndAStats()
Definition: XrdPfcInfo.cc:232
static const char * m_traceID
Definition: XrdPfcInfo.hh:308
int m_astatSize
size of AStat vector
Definition: XrdPfcInfo.hh:84

References CalcCksumStore(), CalcCksumSyncedAndAStats(), Macaroons::Error, GetBitvecSizeInBytes(), m_astats, XrdPfc::Info::Store::m_astatSize, m_buff_synced, m_buff_written, m_store, m_trace, m_traceID, m_version, ResizeBits(), s_defaultVersion, TRACE, UpdateDownloadCompleteStatus(), and Warning.

Referenced by XrdPfc::Cache::DetermineFullFileSize(), and XrdPfc::Cache::LocalFilePath().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ RefAStats()

const std::vector<AStat>& XrdPfc::Info::RefAStats ( ) const
inline

Definition at line 272 of file XrdPfcInfo.hh.

272 { return m_astats; }

References m_astats.

◆ RefStoredData()

const Store& XrdPfc::Info::RefStoredData ( ) const
inline

Get stored data.

Definition at line 271 of file XrdPfcInfo.hh.

271 { return m_store; }

References m_store.

◆ ResetAllAccessStats()

void Info::ResetAllAccessStats ( )

Reset IO Stats.

Definition at line 359 of file XrdPfcInfo.cc.

360 {
361  m_store.m_accessCnt = 0;
362  m_store.m_astatSize = 0;
363  m_astats.clear();
364 }

References XrdPfc::Info::Store::m_accessCnt, m_astats, XrdPfc::Info::Store::m_astatSize, and m_store.

◆ ResetCkSumCache()

void Info::ResetCkSumCache ( )

Definition at line 203 of file XrdPfcInfo.cc.

204 {
205  if (IsCkSumCache())
206  {
207  m_store.m_status.f_cksum_check &= ~CSChk_Cache;
208  if ( ! HasNoCkSumTime())
209  m_store.m_noCkSumTime = time(0);
210  }
211 }
bool IsCkSumCache() const
Definition: XrdPfcInfo.hh:289
bool HasNoCkSumTime() const
Definition: XrdPfcInfo.hh:299

References XrdPfc::CSChk_Cache, HasNoCkSumTime(), IsCkSumCache(), XrdPfc::Info::Store::m_noCkSumTime, XrdPfc::Info::Store::m_status, and m_store.

+ Here is the call graph for this function:

◆ ResetCkSumNet()

void Info::ResetCkSumNet ( )

Definition at line 213 of file XrdPfcInfo.cc.

214 {
215  if (IsCkSumNet())
216  {
217  m_store.m_status.f_cksum_check &= ~CSChk_Net;
218  if ( ! HasNoCkSumTime())
219  m_store.m_noCkSumTime = time(0);
220  }
221 }
bool IsCkSumNet() const
Definition: XrdPfcInfo.hh:290

References XrdPfc::CSChk_Net, HasNoCkSumTime(), IsCkSumNet(), XrdPfc::Info::Store::m_noCkSumTime, XrdPfc::Info::Store::m_status, and m_store.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ ResetNoCkSumTime()

void XrdPfc::Info::ResetNoCkSumTime ( )
inline

Definition at line 302 of file XrdPfcInfo.hh.

302 { m_store.m_noCkSumTime = 0; }

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

◆ ResizeBits()

void Info::ResizeBits ( )

Reserve bit vectors for file_size / buffer_size bytes.

Definition at line 172 of file XrdPfcInfo.cc.

173 {
174  // drop buffer in case of failed/partial reads
175 
176  if (m_buff_synced) free(m_buff_synced);
177  if (m_buff_written) free(m_buff_written);
178  if (m_buff_prefetch) free(m_buff_prefetch);
179 
181 
182  m_buff_written = (unsigned char*) malloc(GetBitvecSizeInBytes());
183  m_buff_synced = (unsigned char*) malloc(GetBitvecSizeInBytes());
184  memset(m_buff_written, 0, GetBitvecSizeInBytes());
185  memset(m_buff_synced, 0, GetBitvecSizeInBytes());
186 
188  m_complete = false;
189 
191  {
192  m_buff_prefetch = (unsigned char*) malloc(GetBitvecSizeInBytes());
194  }
195  else
196  {
197  m_buff_prefetch = 0;
198  }
199 }

References GetBitvecSizeInBytes(), m_bitvecSizeInBits, m_buff_prefetch, m_buff_synced, m_buff_written, XrdPfc::Info::Store::m_buffer_size, m_complete, XrdPfc::Info::Store::m_file_size, m_hasPrefetchBuffer, m_missingBlocks, and m_store.

Referenced by Read(), and SetBufferSizeFileSizeAndCreationTime().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetAllBitsSynced()

void Info::SetAllBitsSynced ( )

Mark all blocks as synced to disk.

Definition at line 144 of file XrdPfcInfo.cc.

145 {
146  // The following should be:
147  // memset(m_buff_synced, 255, GetBitvecSizeInBytes());
148  // but GCC produces an overzealous 'possible argument transpose warning' and
149  // xrootd build uses warnings->errors escalation.
150  // This workaround can be removed for gcc >= 5.
151  // See also: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61294
152  const int nb = GetBitvecSizeInBytes();
153  for (int i = 0; i < nb; ++i)
154  m_buff_synced[i] = 255;
155 
156  m_complete = true;
157 }

References GetBitvecSizeInBytes(), m_buff_synced, and m_complete.

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), and XrdPfc::DataFsSnapshot::write_json_file().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitPrefetch()

void XrdPfc::Info::SetBitPrefetch ( int  i)
inline

Mark block as obtained through prefetch.

Definition at line 365 of file XrdPfcInfo.hh.

366 {
367  if (!m_buff_prefetch) return;
368 
369  const int cn = i/8;
370  assert(cn < GetBitvecSizeInBytes());
371 
372  const int off = i - cn*8;
373  m_buff_prefetch[cn] |= cfiBIT(off);
374 }

References GetBitvecSizeInBytes(), and m_buff_prefetch.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitSynced()

void XrdPfc::Info::SetBitSynced ( int  i)
inline

Mark block as synced to disk.

Definition at line 387 of file XrdPfcInfo.hh.

388 {
389  const int cn = i/8;
390  assert(cn < GetBitvecSizeInBytes());
391 
392  const int off = i - cn*8;
393  m_buff_synced[cn] |= cfiBIT(off);
394 }

References GetBitvecSizeInBytes(), and m_buff_synced.

Referenced by XrdPfc::File::Sync(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitWritten()

void XrdPfc::Info::SetBitWritten ( int  i)
inline

Mark block as written to disk.

Definition at line 352 of file XrdPfcInfo.hh.

353 {
354  const int cn = i/8;
355  assert(cn < GetBitvecSizeInBytes());
356 
357  const int off = i - cn*8;
358 
359  m_buff_written[cn] |= cfiBIT(off);
360 
361  if (--m_missingBlocks == 0)
362  m_complete = true;
363 }

References GetBitvecSizeInBytes(), m_buff_written, m_complete, and m_missingBlocks.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBufferSizeFileSizeAndCreationTime()

void Info::SetBufferSizeFileSizeAndCreationTime ( long long  bs,
long long  fs 
)

Definition at line 161 of file XrdPfcInfo.cc.

162 {
163  // Needed only when Info object is created for the first time in File::Open()
164  m_store.m_buffer_size = bs;
165  m_store.m_file_size = fs;
166  ResizeBits();
167  m_store.m_creationTime = time(0);
168 }

References XrdPfc::Info::Store::m_buffer_size, XrdPfc::Info::Store::m_creationTime, XrdPfc::Info::Store::m_file_size, m_store, and ResizeBits().

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), and XrdPfc::DataFsSnapshot::write_json_file().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetCkSumState()

void XrdPfc::Info::SetCkSumState ( CkSumCheck_e  css)
inline

Definition at line 294 of file XrdPfcInfo.hh.

294 { m_store.m_status.f_cksum_check = css; }

References XrdPfc::Info::Store::m_status, and m_store.

◆ TestBitPrefetch()

bool XrdPfc::Info::TestBitPrefetch ( int  i) const
inline

Test if block at the given index has been prefetched.

Definition at line 376 of file XrdPfcInfo.hh.

377 {
378  if (!m_buff_prefetch) return false;
379 
380  const int cn = i/8;
381  assert(cn < GetBitvecSizeInBytes());
382 
383  const int off = i - cn*8;
384  return (m_buff_prefetch[cn] & cfiBIT(off)) != 0;
385 }

References GetBitvecSizeInBytes(), and m_buff_prefetch.

+ Here is the call graph for this function:

◆ TestBitWritten()

bool XrdPfc::Info::TestBitWritten ( int  i) const
inline

Test if block at the given index is written to disk.

Definition at line 343 of file XrdPfcInfo.hh.

344 {
345  const int cn = i/8;
346  assert(cn < GetBitvecSizeInBytes());
347 
348  const int off = i - cn*8;
349  return (m_buff_written[cn] & cfiBIT(off)) != 0;
350 }

References GetBitvecSizeInBytes(), and m_buff_written.

Referenced by CountBlocksNotWrittenInRng(), GetLastDownloadedBlock(), GetNDownloadedBlocks(), and XrdPfc::File::Prefetch().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ UpdateDownloadCompleteStatus()

void XrdPfc::Info::UpdateDownloadCompleteStatus ( )
inline

Update complete status.

Definition at line 463 of file XrdPfcInfo.hh.

464 {
466  m_complete = (m_missingBlocks == 0);
467 }
int CountBlocksNotWrittenInRng(int firstIdx, int lastIdx) const
Check download status in given block range.
Definition: XrdPfcInfo.hh:452

References CountBlocksNotWrittenInRng(), m_bitvecSizeInBits, m_complete, and m_missingBlocks.

Referenced by Read().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ Write()

bool Info::Write ( XrdOssDF fp,
const char *  dname,
const char *  fname = 0 
)

Write number of blocks and read buffer size

Parameters
fpfile handle
dnamedirectory name for trace output
fnameoptional file name for trace output (can be included in dname)
Returns
true on success

Definition at line 266 of file XrdPfcInfo.cc.

267 {
268  TraceHeader trace_pfx("Write()", dname, fname);
269 
271  m_store.m_astatSize = (int32_t) m_astats.size();
272 
273  FpHelper w(fp, 0, m_trace, m_traceID, trace_pfx);
274 
275  if (w.Write(s_defaultVersion) ||
276  w.Write(m_store) ||
277  w.Write(CalcCksumStore()) ||
278  w.WriteRaw(m_buff_synced, GetBitvecSizeInBytes()) ||
279  w.WriteRaw(m_astats.data(), m_store.m_astatSize * sizeof(AStat)) ||
280  w.Write(CalcCksumSyncedAndAStats()))
281  {
282  return false;
283  }
284 
285  return true;
286 }
void CompactifyAccessRecords()
Compactify access records to the configured maximum.
Definition: XrdPfcInfo.cc:379

References CalcCksumStore(), CalcCksumSyncedAndAStats(), CompactifyAccessRecords(), GetBitvecSizeInBytes(), m_astats, XrdPfc::Info::Store::m_astatSize, m_buff_synced, m_store, m_trace, m_traceID, s_defaultVersion, and s_maxNumAccess.

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), XrdPfc::Cache::LocalFilePath(), XrdPfc::File::Sync(), and XrdPfc::DataFsSnapshot::write_json_file().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ WriteIOStat()

void Info::WriteIOStat ( Stats s)

Write bytes missed, hits, and disk.

Definition at line 429 of file XrdPfcInfo.cc.

430 {
431  m_astats.back().NumIos = s.m_NumIos;
432  m_astats.back().Duration = s.m_Duration;
433  m_astats.back().BytesHit = s.m_BytesHit;
434  m_astats.back().BytesMissed = s.m_BytesMissed;
435  m_astats.back().BytesBypassed = s.m_BytesBypassed;
436 }
long long m_BytesMissed
number of bytes served from remote and cached
Definition: XrdPfcStats.hh:40
long long m_BytesBypassed
number of bytes served directly through XrdCl
Definition: XrdPfcStats.hh:41
int m_Duration
total duration of all IOs attached
Definition: XrdPfcStats.hh:38
int m_NumIos
number of IO objects attached during this access
Definition: XrdPfcStats.hh:37
long long m_BytesHit
number of bytes served from disk
Definition: XrdPfcStats.hh:39

References m_astats, XrdPfc::Stats::m_BytesBypassed, XrdPfc::Stats::m_BytesHit, XrdPfc::Stats::m_BytesMissed, XrdPfc::Stats::m_Duration, and XrdPfc::Stats::m_NumIos.

Referenced by XrdPfc::File::Sync(), and WriteIOStatDetach().

+ Here is the caller graph for this function:

◆ WriteIOStatAttach()

void Info::WriteIOStatAttach ( )

Write open time in the last entry of access statistics.

Definition at line 420 of file XrdPfcInfo.cc.

421 {
423 
424  AStat as;
425  as.AttachTime = time(0);
426  m_astats.push_back(as);
427 }

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::Store::m_accessCnt, m_astats, and m_store.

◆ WriteIOStatDetach()

void Info::WriteIOStatDetach ( Stats s)

Write close time together with bytes missed, hits, and disk.

Definition at line 438 of file XrdPfcInfo.cc.

439 {
440  m_astats.back().DetachTime = time(0);
441  WriteIOStat(s);
442 }
void WriteIOStat(Stats &s)
Write bytes missed, hits, and disk.
Definition: XrdPfcInfo.cc:429

References m_astats, and WriteIOStat().

Referenced by XrdPfc::File::FinalizeSyncBeforeExit().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ WriteIOStatSingle() [1/2]

void Info::WriteIOStatSingle ( long long  bytes_disk)

Write single open/close time for given bytes read from disk.

Definition at line 444 of file XrdPfcInfo.cc.

445 {
447 
448  AStat as;
449  as.AttachTime = as.DetachTime = time(0);
450  as.NumIos = 1;
451  as.BytesHit = bytes_disk;
452  m_astats.push_back(as);
453 }

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::BytesHit, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::Store::m_accessCnt, m_astats, m_store, and XrdPfc::Info::AStat::NumIos.

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), and XrdPfc::Cache::LocalFilePath().

+ Here is the caller graph for this function:

◆ WriteIOStatSingle() [2/2]

void Info::WriteIOStatSingle ( long long  bytes_disk,
time_t  att,
time_t  dtc 
)

Write open/close with given time and bytes read from disk.

Definition at line 455 of file XrdPfcInfo.cc.

456 {
458 
459  AStat as;
460  as.AttachTime = att;
461  as.DetachTime = dtc;
462  as.NumIos = 1;
463  as.Duration = dtc - att;
464  as.BytesHit = bytes_disk;
465  m_astats.push_back(as);
466 }

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::BytesHit, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::AStat::Duration, XrdPfc::Info::Store::m_accessCnt, m_astats, m_store, and XrdPfc::Info::AStat::NumIos.

Member Data Documentation

◆ m_astats

◆ m_bitvecSizeInBits

int XrdPfc::Info::m_bitvecSizeInBits
protected

◆ m_buff_prefetch

unsigned char* XrdPfc::Info::m_buff_prefetch
protected

prefetch statistics

Definition at line 322 of file XrdPfcInfo.hh.

Referenced by ~Info(), ResizeBits(), SetBitPrefetch(), and TestBitPrefetch().

◆ m_buff_synced

unsigned char* XrdPfc::Info::m_buff_synced
protected

disk written state vector

Definition at line 320 of file XrdPfcInfo.hh.

Referenced by ~Info(), CalcCksumSyncedAndAStats(), Read(), ResizeBits(), SetAllBitsSynced(), SetBitSynced(), and Write().

◆ m_buff_written

unsigned char* XrdPfc::Info::m_buff_written
protected

download state vector

Definition at line 321 of file XrdPfcInfo.hh.

Referenced by ~Info(), Read(), ResizeBits(), SetBitWritten(), and TestBitWritten().

◆ m_complete

bool XrdPfc::Info::m_complete
protected

cached; if false, set to true when missingBlocks hit zero

Definition at line 328 of file XrdPfcInfo.hh.

Referenced by IsComplete(), ResizeBits(), SetAllBitsSynced(), SetBitWritten(), and UpdateDownloadCompleteStatus().

◆ m_hasPrefetchBuffer

bool XrdPfc::Info::m_hasPrefetchBuffer
protected

constains current prefetch score

Definition at line 329 of file XrdPfcInfo.hh.

Referenced by ResizeBits().

◆ m_missingBlocks

int XrdPfc::Info::m_missingBlocks
protected

cached, updated in SetBitWritten()

Definition at line 327 of file XrdPfcInfo.hh.

Referenced by ResizeBits(), SetBitWritten(), and UpdateDownloadCompleteStatus().

◆ m_store

◆ m_trace

XrdSysTrace* XrdPfc::Info::m_trace
protected

Definition at line 317 of file XrdPfcInfo.hh.

Referenced by GetTrace(), Read(), and Write().

◆ m_traceID

const char * Info::m_traceID = "CInfo"
static

Definition at line 308 of file XrdPfcInfo.hh.

Referenced by Read(), and Write().

◆ m_version

int XrdPfc::Info::m_version
protected

Definition at line 325 of file XrdPfcInfo.hh.

Referenced by GetVersion(), and Read().

◆ s_defaultVersion

const int Info::s_defaultVersion = 4
static

Definition at line 312 of file XrdPfcInfo.hh.

Referenced by Read(), and Write().

◆ s_infoExtension

◆ s_infoExtensionLen

const size_t Info::s_infoExtensionLen = strlen(Info::s_infoExtension)
static

◆ s_maxNumAccess

size_t Info::s_maxNumAccess = 20
static

Definition at line 311 of file XrdPfcInfo.hh.

Referenced by CompactifyAccessRecords(), XrdPfc::Cache::Config(), and Write().


The documentation for this class was generated from the following files: