XRootD
XrdPfc::Info Class Reference

Status of cached file. Can be read from and written into a binary file. More...

#include <XrdPfcInfo.hh>

+ Collaboration diagram for XrdPfc::Info:

Classes

struct  AStat
 Access statistics. More...
 
struct  Status
 
union  Status.__unnamed126__
 
struct  Status.__unnamed126__.__unnamed128__
 
struct  Store
 

Public Member Functions

 Info (XrdSysTrace *trace, bool prefetchBuffer=false)
 Constructor. More...
 
 ~Info ()
 Destructor. More...
 
void CalcCksumMd5 (unsigned char *buff, char *digest)
 
uint32_t CalcCksumStore ()
 Get cksum, MD5 is for backward compatibility with V2 and V3. More...
 
uint32_t CalcCksumSyncedAndAStats ()
 
void CompactifyAccessRecords ()
 Compactify access records to the configured maximum. More...
 
int CountBlocksNotWrittenInRng (int firstIdx, int lastIdx) const
 Check download status in given block range. More...
 
void DowngradeCkSumState (CkSumCheck_e css_ref)
 
size_t GetAccessCnt () const
 Get number of accesses. More...
 
int GetBitvecSizeInBytes () const
 Get size of download-state bit-vector in bytes. More...
 
long long GetBufferSize () const
 Get prefetch buffer size. More...
 
CkSumCheck_e GetCkSumState () const
 
const char * GetCkSumStateAsText () const
 
time_t GetCreationTime () const
 Get file size. More...
 
long long GetExpectedDataFileSize () const
 Get expected data file size. More...
 
long long GetFileSize () const
 Get file size. More...
 
const AStatGetLastAccessStats () const
 Get latest access stats. More...
 
int GetLastDownloadedBlock () const
 Get number of the last downloaded block. More...
 
bool GetLatestDetachTime (time_t &t) const
 Get latest detach time. More...
 
int GetNBlocks () const
 Get number of blocks represented in download-state bit-vector. More...
 
int GetNDownloadedBlocks () const
 Get number of downloaded blocks. More...
 
long long GetNDownloadedBytes () const
 Get number of downloaded bytes. More...
 
time_t GetNoCkSumTime () const
 
time_t GetNoCkSumTimeForUVKeep () const
 
XrdSysTraceGetTrace () const
 
int GetVersion ()
 Get version. More...
 
bool HasNoCkSumTime () const
 
bool IsCkSumAny () const
 
bool IsCkSumBoth () const
 
bool IsCkSumCache () const
 
bool IsCkSumNet () const
 
bool IsComplete () const
 Get complete status. More...
 
bool Read (XrdOssDF *fp, const char *dname, const char *fname=0)
 Read content of cinfo file into this object. More...
 
const std::vector< AStat > & RefAStats () const
 
const StoreRefStoredData () const
 Get stored data. More...
 
void ResetAllAccessStats ()
 Reset IO Stats. More...
 
void ResetCkSumCache ()
 
void ResetCkSumNet ()
 
void ResetNoCkSumTime ()
 
void ResizeBits ()
 Reserve bit vectors for file_size / buffer_size bytes. More...
 
void SetAllBitsSynced ()
 Mark all blocks as synced to disk. More...
 
void SetBitPrefetch (int i)
 Mark block as obtained through prefetch. More...
 
void SetBitSynced (int i)
 Mark block as synced to disk. More...
 
void SetBitWritten (int i)
 Mark block as written to disk. More...
 
void SetBufferSizeFileSizeAndCreationTime (long long bs, long long fs)
 
void SetCkSumState (CkSumCheck_e css)
 
bool TestBitPrefetch (int i) const
 Test if block at the given index has been prefetched. More...
 
bool TestBitWritten (int i) const
 Test if block at the given index is written to disk. More...
 
void UpdateDownloadCompleteStatus ()
 Update complete status. More...
 
bool Write (XrdOssDF *fp, const char *dname, const char *fname=0)
 
void WriteIOStat (Stats &s)
 Write bytes missed, hits, and disk. More...
 
void WriteIOStatAttach ()
 Write open time in the last entry of access statistics. More...
 
void WriteIOStatDetach (Stats &s)
 Write close time together with bytes missed, hits, and disk. More...
 
void WriteIOStatSingle (long long bytes_disk)
 Write single open/close time for given bytes read from disk. More...
 
void WriteIOStatSingle (long long bytes_disk, time_t att, time_t dtc)
 Write open/close with given time and bytes read from disk. More...
 

Static Public Attributes

static const char * m_traceID = "CInfo"
 
static const int s_defaultVersion = 4
 
static const char * s_infoExtension = ".cinfo"
 
static const size_t s_infoExtensionLen = strlen(Info::s_infoExtension)
 
static size_t s_maxNumAccess = 20
 

Protected Attributes

std::vector< AStatm_astats
 access records More...
 
int m_bitvecSizeInBits
 cached More...
 
unsigned char * m_buff_prefetch
 prefetch statistics More...
 
unsigned char * m_buff_synced
 disk written state vector More...
 
unsigned char * m_buff_written
 download state vector More...
 
bool m_complete
 cached; if false, set to true when missingBlocks hit zero More...
 
bool m_hasPrefetchBuffer
 constains current prefetch score More...
 
int m_missingBlocks
 cached, updated in SetBitWritten() More...
 
Store m_store
 
XrdSysTracem_trace
 
int m_version
 

Detailed Description

Status of cached file. Can be read from and written into a binary file.

Definition at line 40 of file XrdPfcInfo.hh.


Class Documentation

◆ XrdPfc::Info::Status.__unnamed126__

union XrdPfc::Info::Status.__unnamed126__

Definition at line 44 of file XrdPfcInfo.hh.

+ Collaboration diagram for XrdPfc::Info::Status.__unnamed126__:
Class Members
__unnamed126__ __unnamed__
unsigned int _raw_

◆ XrdPfc::Info::Status.__unnamed126__.__unnamed128__

struct XrdPfc::Info::Status.__unnamed126__.__unnamed128__

Definition at line 45 of file XrdPfcInfo.hh.

+ Collaboration diagram for XrdPfc::Info::Status.__unnamed126__.__unnamed128__:
Class Members
int _free_bits_: 29
int f_cksum_check: 3 as in enum CkSumCheck_e

Constructor & Destructor Documentation

◆ Info()

Info::Info ( XrdSysTrace trace,
bool  prefetchBuffer = false 
)

Constructor.

Definition at line 125 of file XrdPfcInfo.cc.

125  :
126  m_trace(trace),
128  m_version(0),
130  m_missingBlocks(0),
131  m_complete(false),
132  m_hasPrefetchBuffer(prefetchBuffer),
133  m_cksCalcMd5(0)
134 {}
unsigned char * m_buff_prefetch
prefetch statistics
Definition: XrdPfcInfo.hh:322
unsigned char * m_buff_written
download state vector
Definition: XrdPfcInfo.hh:321
unsigned char * m_buff_synced
disk written state vector
Definition: XrdPfcInfo.hh:320
bool m_complete
cached; if false, set to true when missingBlocks hit zero
Definition: XrdPfcInfo.hh:328
int m_missingBlocks
cached, updated in SetBitWritten()
Definition: XrdPfcInfo.hh:327
int m_bitvecSizeInBits
cached
Definition: XrdPfcInfo.hh:326
XrdSysTrace * m_trace
Definition: XrdPfcInfo.hh:317
bool m_hasPrefetchBuffer
constains current prefetch score
Definition: XrdPfcInfo.hh:329

◆ ~Info()

Info::~Info ( )

Destructor.

Definition at line 136 of file XrdPfcInfo.cc.

137 {
138  if (m_buff_synced) free(m_buff_synced);
139  if (m_buff_written) free(m_buff_written);
140  if (m_buff_prefetch) free(m_buff_prefetch);
141  delete m_cksCalcMd5;
142 }

References m_buff_prefetch, m_buff_synced, and m_buff_written.

Member Function Documentation

◆ CalcCksumMd5()

void Info::CalcCksumMd5 ( unsigned char *  buff,
char *  digest 
)

Definition at line 240 of file XrdPfcInfo.cc.

241 {
242  if (m_cksCalcMd5)
243  m_cksCalcMd5->Init();
244  else
245  m_cksCalcMd5 = new XrdCksCalcmd5();
246 
247  m_cksCalcMd5->Update((const char*)buff, GetBitvecSizeInBytes());
248  memcpy(digest, m_cksCalcMd5->Final(), 16);
249 }
virtual char * Final()=0
virtual void Update(const char *Buff, int BLen)=0
virtual void Init()=0
int GetBitvecSizeInBytes() const
Get size of download-state bit-vector in bytes.
Definition: XrdPfcInfo.hh:429

References XrdCksCalc::Final(), GetBitvecSizeInBytes(), XrdCksCalc::Init(), and XrdCksCalc::Update().

+ Here is the call graph for this function:

◆ CalcCksumStore()

uint32_t Info::CalcCksumStore ( )

Get cksum, MD5 is for backward compatibility with V2 and V3.

Definition at line 229 of file XrdPfcInfo.cc.

230 {
231  return crc32c(0, &m_store, sizeof(Store));
232 }
uint32_t crc32c(uint32_t crc, void const *buf, size_t len)

References crc32c(), and m_store.

Referenced by Read(), and Write().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ CalcCksumSyncedAndAStats()

uint32_t Info::CalcCksumSyncedAndAStats ( )

Definition at line 234 of file XrdPfcInfo.cc.

235 {
236  uint32_t cks = crc32c(0, m_buff_synced, GetBitvecSizeInBytes());
237  return crc32c(cks, m_astats.data(), m_astats.size() * sizeof(AStat));
238 }
std::vector< AStat > m_astats
access records
Definition: XrdPfcInfo.hh:323

References crc32c(), GetBitvecSizeInBytes(), m_astats, and m_buff_synced.

Referenced by Read(), and Write().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ CompactifyAccessRecords()

void Info::CompactifyAccessRecords ( )

Compactify access records to the configured maximum.

Definition at line 381 of file XrdPfcInfo.cc.

382 {
383  time_t now = time(0);
384 
385  std::vector<AStat> &v = m_astats;
386 
387  for (int i = 0; i < (int) v.size() - 1; ++i)
388  {
389  if (v[i].DetachTime == 0)
390  v[i].DetachTime = std::min(v[i].AttachTime + v[i].Duration / v[i].NumIos, v[i+1].AttachTime);
391  }
392 
393  while (v.size() > s_maxNumAccess)
394  {
395  double min_s = 1e10;
396  int min_i = -1;
397 
398  int M = (int) v.size() - 2;
399  for (int i = 0; i < M; ++i)
400  {
401  AStat &a = v[i], &b = v[i + 1];
402 
403  time_t t = std::max((time_t) 1, (now - b.AttachTime) / 2 + (now - a.DetachTime) / 2);
404  double s = (double) (b.AttachTime - a.DetachTime) / t;
405 
406  if (s < min_s)
407  {
408  min_s = s;
409  min_i = i;
410  }
411  }
412  assert(min_i != -1);
413 
414  v[min_i].MergeWith(v[min_i + 1]);
415 
416  v.erase(v.begin() + (min_i + 1));
417  }
418 }
static size_t s_maxNumAccess
Definition: XrdPfcInfo.hh:311

References XrdPfc::Info::AStat::DetachTime, m_astats, and s_maxNumAccess.

Referenced by Write().

+ Here is the caller graph for this function:

◆ CountBlocksNotWrittenInRng()

int XrdPfc::Info::CountBlocksNotWrittenInRng ( int  firstIdx,
int  lastIdx 
) const
inline

Check download status in given block range.

Definition at line 452 of file XrdPfcInfo.hh.

453 {
454  // TODO rewrite to use full byte comparisons outside of edges ?
455  // Also, it seems to be always called with firstIdx = 0, lastIdx = m_bitvecSizeInBits.
456  int cnt = 0;
457  for (int i = firstIdx; i < lastIdx; ++i)
458  if (! TestBitWritten(i)) ++cnt;
459 
460  return cnt;
461 }
bool TestBitWritten(int i) const
Test if block at the given index is written to disk.
Definition: XrdPfcInfo.hh:343

References TestBitWritten().

Referenced by UpdateDownloadCompleteStatus().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ DowngradeCkSumState()

void XrdPfc::Info::DowngradeCkSumState ( CkSumCheck_e  css_ref)
inline

Definition at line 295 of file XrdPfcInfo.hh.

295 { m_store.m_status.f_cksum_check &= css_ref; }
Status m_status
status information
Definition: XrdPfcInfo.hh:83

References XrdPfc::Info::Store::m_status, and m_store.

◆ GetAccessCnt()

size_t XrdPfc::Info::GetAccessCnt ( ) const
inline

Get number of accesses.

Definition at line 261 of file XrdPfcInfo.hh.

261 { return m_store.m_accessCnt; }
size_t m_accessCnt
total access count for the file
Definition: XrdPfcInfo.hh:82

References XrdPfc::Info::Store::m_accessCnt, and m_store.

Referenced by XrdPfc::File::GetAccessCnt().

+ Here is the caller graph for this function:

◆ GetBitvecSizeInBytes()

int XrdPfc::Info::GetBitvecSizeInBytes ( ) const
inline

Get size of download-state bit-vector in bytes.

Definition at line 429 of file XrdPfcInfo.hh.

430 {
431  if (m_bitvecSizeInBits)
432  return ((m_bitvecSizeInBits - 1)/8 + 1);
433  else
434  return 0;
435 }

References m_bitvecSizeInBits.

Referenced by CalcCksumMd5(), CalcCksumSyncedAndAStats(), Read(), ResizeBits(), SetAllBitsSynced(), SetBitPrefetch(), SetBitSynced(), SetBitWritten(), TestBitPrefetch(), TestBitWritten(), and Write().

+ Here is the caller graph for this function:

◆ GetBufferSize()

long long XrdPfc::Info::GetBufferSize ( ) const
inline

Get prefetch buffer size.

Definition at line 469 of file XrdPfcInfo.hh.

470 {
471  return m_store.m_buffer_size;
472 }
long long m_buffer_size
buffer / block size
Definition: XrdPfcInfo.hh:78

References XrdPfc::Info::Store::m_buffer_size, and m_store.

Referenced by XrdPfc::File::GetBlockSize().

+ Here is the caller graph for this function:

◆ GetCkSumState()

CkSumCheck_e XrdPfc::Info::GetCkSumState ( ) const
inline

Definition at line 286 of file XrdPfcInfo.hh.

286 { return (CkSumCheck_e) m_store.m_status.f_cksum_check; }

References XrdPfc::Info::Store::m_status, and m_store.

Referenced by XrdPfc::FPurgeState::CheckFile().

+ Here is the caller graph for this function:

◆ GetCkSumStateAsText()

const char * Info::GetCkSumStateAsText ( ) const

Definition at line 251 of file XrdPfcInfo.cc.

252 {
253  switch (m_store.m_status.f_cksum_check) {
254  case CSChk_None : return "none";
255  case CSChk_Cache : return "cache";
256  case CSChk_Net : return "net";
257  case CSChk_Both : return "both";
258  default : return "unknown";
259  }
260 }
@ CSChk_Both
Definition: XrdPfcTypes.hh:27
@ CSChk_Net
Definition: XrdPfcTypes.hh:27
@ CSChk_Cache
Definition: XrdPfcTypes.hh:27
@ CSChk_None
Definition: XrdPfcTypes.hh:27

References XrdPfc::CSChk_Both, XrdPfc::CSChk_Cache, XrdPfc::CSChk_Net, XrdPfc::CSChk_None, XrdPfc::Info::Store::m_status, and m_store.

◆ GetCreationTime()

time_t XrdPfc::Info::GetCreationTime ( ) const
inline

Get file size.

Definition at line 277 of file XrdPfcInfo.hh.

277 { return m_store.m_creationTime; }
time_t m_creationTime
time the info file was created
Definition: XrdPfcInfo.hh:80

References XrdPfc::Info::Store::m_creationTime, and m_store.

◆ GetExpectedDataFileSize()

long long XrdPfc::Info::GetExpectedDataFileSize ( ) const
inline

Get expected data file size.

Definition at line 420 of file XrdPfcInfo.hh.

421 {
422  int last_block = GetLastDownloadedBlock();
423  if (last_block == m_bitvecSizeInBits - 1)
424  return m_store.m_file_size;
425  else
426  return (last_block + 1) * m_store.m_buffer_size;
427 }
int GetLastDownloadedBlock() const
Get number of the last downloaded block.
Definition: XrdPfcInfo.hh:412
long long m_file_size
size of file in bytes
Definition: XrdPfcInfo.hh:79

References GetLastDownloadedBlock(), m_bitvecSizeInBits, XrdPfc::Info::Store::m_buffer_size, XrdPfc::Info::Store::m_file_size, and m_store.

+ Here is the call graph for this function:

◆ GetFileSize()

long long XrdPfc::Info::GetFileSize ( ) const
inline

Get file size.

Definition at line 442 of file XrdPfcInfo.hh.

443 {
444  return m_store.m_file_size;
445 }

References XrdPfc::Info::Store::m_file_size, and m_store.

Referenced by XrdPfc::Cache::DetermineFullFileSize(), and XrdPfc::Cache::LocalFilePath().

+ Here is the caller graph for this function:

◆ GetLastAccessStats()

const Info::AStat * Info::GetLastAccessStats ( ) const

Get latest access stats.

Definition at line 491 of file XrdPfcInfo.cc.

492 {
493  return m_astats.empty() ? 0 : & m_astats.back();
494 }

References m_astats.

Referenced by XrdPfc::File::GetLastAccessStats().

+ Here is the caller graph for this function:

◆ GetLastDownloadedBlock()

int XrdPfc::Info::GetLastDownloadedBlock ( ) const
inline

Get number of the last downloaded block.

Definition at line 412 of file XrdPfcInfo.hh.

413 {
414  for (int i = m_bitvecSizeInBits - 1; i >= 0; --i)
415  if (TestBitWritten(i)) return i;
416 
417  return -1;
418 }

References m_bitvecSizeInBits, and TestBitWritten().

Referenced by GetExpectedDataFileSize().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ GetLatestDetachTime()

bool Info::GetLatestDetachTime ( time_t &  t) const

Get latest detach time.

Definition at line 472 of file XrdPfcInfo.cc.

473 {
474  if (m_astats.empty())
475  {
477  }
478  else
479  {
480  const AStat& ls = m_astats.back();
481 
482  if (ls.DetachTime == 0)
483  t = ls.AttachTime + ls.Duration;
484  else
485  t = ls.DetachTime;
486  }
487 
488  return t != 0;
489 }

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::AStat::Duration, m_astats, XrdPfc::Info::Store::m_creationTime, and m_store.

Referenced by XrdPfc::FPurgeState::CheckFile(), and XrdPfc::Cache::ExecuteCommandUrl().

+ Here is the caller graph for this function:

◆ GetNBlocks()

int XrdPfc::Info::GetNBlocks ( ) const
inline

Get number of blocks represented in download-state bit-vector.

Definition at line 437 of file XrdPfcInfo.hh.

438 {
439  return m_bitvecSizeInBits;
440 }

References m_bitvecSizeInBits.

Referenced by XrdPfc::File::GetNBlocks().

+ Here is the caller graph for this function:

◆ GetNDownloadedBlocks()

int XrdPfc::Info::GetNDownloadedBlocks ( ) const
inline

Get number of downloaded blocks.

Definition at line 398 of file XrdPfcInfo.hh.

399 {
400  int cntd = 0;
401  for (int i = 0; i < m_bitvecSizeInBits; ++i)
402  if (TestBitWritten(i)) cntd++;
403 
404  return cntd;
405 }

References m_bitvecSizeInBits, and TestBitWritten().

Referenced by XrdPfc::File::GetNDownloadedBlocks(), and GetNDownloadedBytes().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ GetNDownloadedBytes()

long long XrdPfc::Info::GetNDownloadedBytes ( ) const
inline

Get number of downloaded bytes.

Definition at line 407 of file XrdPfcInfo.hh.

408 {
410 }
int GetNDownloadedBlocks() const
Get number of downloaded blocks.
Definition: XrdPfcInfo.hh:398

References GetNDownloadedBlocks(), XrdPfc::Info::Store::m_buffer_size, and m_store.

+ Here is the call graph for this function:

◆ GetNoCkSumTime()

time_t XrdPfc::Info::GetNoCkSumTime ( ) const
inline

Definition at line 300 of file XrdPfcInfo.hh.

300 { return m_store.m_noCkSumTime; }
time_t m_noCkSumTime
time when first non-cksummed block was detected
Definition: XrdPfcInfo.hh:81

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

◆ GetNoCkSumTimeForUVKeep()

time_t XrdPfc::Info::GetNoCkSumTimeForUVKeep ( ) const
inline

Definition at line 301 of file XrdPfcInfo.hh.

References XrdPfc::Info::Store::m_creationTime, XrdPfc::Info::Store::m_noCkSumTime, and m_store.

Referenced by XrdPfc::FPurgeState::CheckFile().

+ Here is the caller graph for this function:

◆ GetTrace()

XrdSysTrace* XrdPfc::Info::GetTrace ( ) const
inline

Definition at line 314 of file XrdPfcInfo.hh.

314 {return m_trace; }

References m_trace.

◆ GetVersion()

int XrdPfc::Info::GetVersion ( )
inline

Get version.

Definition at line 266 of file XrdPfcInfo.hh.

266 { return m_version; }

References m_version.

◆ HasNoCkSumTime()

bool XrdPfc::Info::HasNoCkSumTime ( ) const
inline

Definition at line 299 of file XrdPfcInfo.hh.

299 { return m_store.m_noCkSumTime != 0; }

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

Referenced by ResetCkSumCache(), and ResetCkSumNet().

+ Here is the caller graph for this function:

◆ IsCkSumAny()

bool XrdPfc::Info::IsCkSumAny ( ) const
inline

Definition at line 291 of file XrdPfcInfo.hh.

291 { return m_store.m_status.f_cksum_check & CSChk_Both; }

References XrdPfc::CSChk_Both, XrdPfc::Info::Store::m_status, and m_store.

◆ IsCkSumBoth()

bool XrdPfc::Info::IsCkSumBoth ( ) const
inline

Definition at line 292 of file XrdPfcInfo.hh.

292 { return (m_store.m_status.f_cksum_check & CSChk_Both) == CSChk_Both; }

References XrdPfc::CSChk_Both, XrdPfc::Info::Store::m_status, and m_store.

◆ IsCkSumCache()

bool XrdPfc::Info::IsCkSumCache ( ) const
inline

Definition at line 289 of file XrdPfcInfo.hh.

289 { return m_store.m_status.f_cksum_check & CSChk_Cache; }

References XrdPfc::CSChk_Cache, XrdPfc::Info::Store::m_status, and m_store.

Referenced by ResetCkSumCache(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ IsCkSumNet()

bool XrdPfc::Info::IsCkSumNet ( ) const
inline

Definition at line 290 of file XrdPfcInfo.hh.

290 { return m_store.m_status.f_cksum_check & CSChk_Net; }

References XrdPfc::CSChk_Net, XrdPfc::Info::Store::m_status, and m_store.

Referenced by ResetCkSumNet(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ IsComplete()

bool XrdPfc::Info::IsComplete ( ) const
inline

Get complete status.

Definition at line 447 of file XrdPfcInfo.hh.

448 {
449  return m_complete;
450 }

References m_complete.

Referenced by XrdPfc::Cache::LocalFilePath(), XrdPfc::File::Read(), XrdPfc::File::ReadV(), XrdPfc::File::Sync(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the caller graph for this function:

◆ Read()

bool Info::Read ( XrdOssDF fp,
const char *  dname,
const char *  fname = 0 
)

Read content of cinfo file into this object.

Parameters
fpfile handle
dnamedirectory name for trace output
fnameoptional file name for trace output (can be included in dname)
Returns
true on success

Definition at line 296 of file XrdPfcInfo.cc.

297 {
298  // Does not need lock, called only in File::Open before File::Run() starts.
299  // XXXX Wait, how about Purge, and LocalFilePath, Stat?
300 
301  TraceHeader trace_pfx("Read()", dname, fname);
302 
303  FpHelper r(fp, 0, m_trace, m_traceID, trace_pfx);
304 
305  if (r.Read(m_version)) return false;
306 
308  {
309  if (m_version == 2)
310  {
311  return ReadV2(fp, r.f_off, dname, fname);
312  }
313  else if (m_version == 3)
314  {
315  return ReadV3(fp, r.f_off, dname, fname);
316  }
317  else
318  {
319  TRACE(Warning, trace_pfx << "File version " << m_version << " not supported.");
320  return false;
321  }
322  }
323 
324  uint32_t cksum;
325 
326  if (r.Read(m_store) || r.Read(cksum)) return false;
327 
328  if (cksum != CalcCksumStore())
329  {
330  TRACE(Error, trace_pfx << "Checksum Store mismatch.");
331  return false;
332  }
333 
334  ResizeBits();
335  m_astats.resize(m_store.m_astatSize);
336 
337  if (r.ReadRaw(m_buff_synced, GetBitvecSizeInBytes()) ||
338  r.ReadRaw(m_astats.data(), m_store.m_astatSize * sizeof(AStat)) ||
339  r.Read(cksum))
340  {
341  return false;
342  }
343 
344  if (cksum != CalcCksumSyncedAndAStats())
345  {
346  TRACE(Error, trace_pfx << "Checksum Synced or AStats mismatch.");
347  return false;
348  }
349 
351 
353 
354  return true;
355 }
@ Warning
#define TRACE(act, x)
Definition: XrdTrace.hh:63
void UpdateDownloadCompleteStatus()
Update complete status.
Definition: XrdPfcInfo.hh:463
uint32_t CalcCksumStore()
Get cksum, MD5 is for backward compatibility with V2 and V3.
Definition: XrdPfcInfo.cc:229
void ResizeBits()
Reserve bit vectors for file_size / buffer_size bytes.
Definition: XrdPfcInfo.cc:174
static const int s_defaultVersion
Definition: XrdPfcInfo.hh:312
uint32_t CalcCksumSyncedAndAStats()
Definition: XrdPfcInfo.cc:234
static const char * m_traceID
Definition: XrdPfcInfo.hh:308
int m_astatSize
size of AStat vector
Definition: XrdPfcInfo.hh:84

References CalcCksumStore(), CalcCksumSyncedAndAStats(), Macaroons::Error, GetBitvecSizeInBytes(), m_astats, XrdPfc::Info::Store::m_astatSize, m_buff_synced, m_buff_written, m_store, m_trace, m_traceID, m_version, ResizeBits(), s_defaultVersion, TRACE, UpdateDownloadCompleteStatus(), and Warning.

Referenced by XrdPfc::Cache::DetermineFullFileSize(), XrdPfc::Cache::LocalFilePath(), and XrdPfc::FPurgeState::ProcessDirAndRecurse().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ RefAStats()

const std::vector<AStat>& XrdPfc::Info::RefAStats ( ) const
inline

Definition at line 272 of file XrdPfcInfo.hh.

272 { return m_astats; }

References m_astats.

◆ RefStoredData()

const Store& XrdPfc::Info::RefStoredData ( ) const
inline

Get stored data.

Definition at line 271 of file XrdPfcInfo.hh.

271 { return m_store; }

References m_store.

◆ ResetAllAccessStats()

void Info::ResetAllAccessStats ( )

Reset IO Stats.

Definition at line 361 of file XrdPfcInfo.cc.

362 {
363  m_store.m_accessCnt = 0;
364  m_store.m_astatSize = 0;
365  m_astats.clear();
366 }

References XrdPfc::Info::Store::m_accessCnt, m_astats, XrdPfc::Info::Store::m_astatSize, and m_store.

◆ ResetCkSumCache()

void Info::ResetCkSumCache ( )

Definition at line 205 of file XrdPfcInfo.cc.

206 {
207  if (IsCkSumCache())
208  {
209  m_store.m_status.f_cksum_check &= ~CSChk_Cache;
210  if ( ! HasNoCkSumTime())
211  m_store.m_noCkSumTime = time(0);
212  }
213 }
bool IsCkSumCache() const
Definition: XrdPfcInfo.hh:289
bool HasNoCkSumTime() const
Definition: XrdPfcInfo.hh:299

References XrdPfc::CSChk_Cache, HasNoCkSumTime(), IsCkSumCache(), XrdPfc::Info::Store::m_noCkSumTime, XrdPfc::Info::Store::m_status, and m_store.

+ Here is the call graph for this function:

◆ ResetCkSumNet()

void Info::ResetCkSumNet ( )

Definition at line 215 of file XrdPfcInfo.cc.

216 {
217  if (IsCkSumNet())
218  {
219  m_store.m_status.f_cksum_check &= ~CSChk_Net;
220  if ( ! HasNoCkSumTime())
221  m_store.m_noCkSumTime = time(0);
222  }
223 }
bool IsCkSumNet() const
Definition: XrdPfcInfo.hh:290

References XrdPfc::CSChk_Net, HasNoCkSumTime(), IsCkSumNet(), XrdPfc::Info::Store::m_noCkSumTime, XrdPfc::Info::Store::m_status, and m_store.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ ResetNoCkSumTime()

void XrdPfc::Info::ResetNoCkSumTime ( )
inline

Definition at line 302 of file XrdPfcInfo.hh.

302 { m_store.m_noCkSumTime = 0; }

References XrdPfc::Info::Store::m_noCkSumTime, and m_store.

◆ ResizeBits()

void Info::ResizeBits ( )

Reserve bit vectors for file_size / buffer_size bytes.

Definition at line 174 of file XrdPfcInfo.cc.

175 {
176  // drop buffer in case of failed/partial reads
177 
178  if (m_buff_synced) free(m_buff_synced);
179  if (m_buff_written) free(m_buff_written);
180  if (m_buff_prefetch) free(m_buff_prefetch);
181 
183 
184  m_buff_written = (unsigned char*) malloc(GetBitvecSizeInBytes());
185  m_buff_synced = (unsigned char*) malloc(GetBitvecSizeInBytes());
186  memset(m_buff_written, 0, GetBitvecSizeInBytes());
187  memset(m_buff_synced, 0, GetBitvecSizeInBytes());
188 
190  m_complete = false;
191 
193  {
194  m_buff_prefetch = (unsigned char*) malloc(GetBitvecSizeInBytes());
196  }
197  else
198  {
199  m_buff_prefetch = 0;
200  }
201 }

References GetBitvecSizeInBytes(), m_bitvecSizeInBits, m_buff_prefetch, m_buff_synced, m_buff_written, XrdPfc::Info::Store::m_buffer_size, m_complete, XrdPfc::Info::Store::m_file_size, m_hasPrefetchBuffer, m_missingBlocks, and m_store.

Referenced by Read(), and SetBufferSizeFileSizeAndCreationTime().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetAllBitsSynced()

void Info::SetAllBitsSynced ( )

Mark all blocks as synced to disk.

Definition at line 146 of file XrdPfcInfo.cc.

147 {
148  // The following should be:
149  // memset(m_buff_synced, 255, GetBitvecSizeInBytes());
150  // but GCC produces an overzealous 'possible argument transpose warning' and
151  // xrootd build uses warnings->errors escalation.
152  // This workaround can be removed for gcc >= 5.
153  // See also: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61294
154  const int nb = GetBitvecSizeInBytes();
155  for (int i = 0; i < nb; ++i)
156  m_buff_synced[i] = 255;
157 
158  m_complete = true;
159 }

References GetBitvecSizeInBytes(), m_buff_synced, and m_complete.

Referenced by XrdPfc::Cache::ExecuteCommandUrl().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitPrefetch()

void XrdPfc::Info::SetBitPrefetch ( int  i)
inline

Mark block as obtained through prefetch.

Definition at line 365 of file XrdPfcInfo.hh.

366 {
367  if (!m_buff_prefetch) return;
368 
369  const int cn = i/8;
370  assert(cn < GetBitvecSizeInBytes());
371 
372  const int off = i - cn*8;
373  m_buff_prefetch[cn] |= cfiBIT(off);
374 }

References GetBitvecSizeInBytes(), and m_buff_prefetch.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitSynced()

void XrdPfc::Info::SetBitSynced ( int  i)
inline

Mark block as synced to disk.

Definition at line 387 of file XrdPfcInfo.hh.

388 {
389  const int cn = i/8;
390  assert(cn < GetBitvecSizeInBytes());
391 
392  const int off = i - cn*8;
393  m_buff_synced[cn] |= cfiBIT(off);
394 }

References GetBitvecSizeInBytes(), and m_buff_synced.

Referenced by XrdPfc::File::Sync(), and XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBitWritten()

void XrdPfc::Info::SetBitWritten ( int  i)
inline

Mark block as written to disk.

Definition at line 352 of file XrdPfcInfo.hh.

353 {
354  const int cn = i/8;
355  assert(cn < GetBitvecSizeInBytes());
356 
357  const int off = i - cn*8;
358 
359  m_buff_written[cn] |= cfiBIT(off);
360 
361  if (--m_missingBlocks == 0)
362  m_complete = true;
363 }

References GetBitvecSizeInBytes(), m_buff_written, m_complete, and m_missingBlocks.

Referenced by XrdPfc::File::WriteBlockToDisk().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetBufferSizeFileSizeAndCreationTime()

void Info::SetBufferSizeFileSizeAndCreationTime ( long long  bs,
long long  fs 
)

Definition at line 163 of file XrdPfcInfo.cc.

164 {
165  // Needed only when Info object is created for the first time in File::Open()
166  m_store.m_buffer_size = bs;
167  m_store.m_file_size = fs;
168  ResizeBits();
169  m_store.m_creationTime = time(0);
170 }

References XrdPfc::Info::Store::m_buffer_size, XrdPfc::Info::Store::m_creationTime, XrdPfc::Info::Store::m_file_size, m_store, and ResizeBits().

Referenced by XrdPfc::Cache::ExecuteCommandUrl().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetCkSumState()

void XrdPfc::Info::SetCkSumState ( CkSumCheck_e  css)
inline

Definition at line 294 of file XrdPfcInfo.hh.

294 { m_store.m_status.f_cksum_check = css; }

References XrdPfc::Info::Store::m_status, and m_store.

◆ TestBitPrefetch()

bool XrdPfc::Info::TestBitPrefetch ( int  i) const
inline

Test if block at the given index has been prefetched.

Definition at line 376 of file XrdPfcInfo.hh.

377 {
378  if (!m_buff_prefetch) return false;
379 
380  const int cn = i/8;
381  assert(cn < GetBitvecSizeInBytes());
382 
383  const int off = i - cn*8;
384  return (m_buff_prefetch[cn] & cfiBIT(off)) != 0;
385 }

References GetBitvecSizeInBytes(), and m_buff_prefetch.

+ Here is the call graph for this function:

◆ TestBitWritten()

bool XrdPfc::Info::TestBitWritten ( int  i) const
inline

Test if block at the given index is written to disk.

Definition at line 343 of file XrdPfcInfo.hh.

344 {
345  const int cn = i/8;
346  assert(cn < GetBitvecSizeInBytes());
347 
348  const int off = i - cn*8;
349  return (m_buff_written[cn] & cfiBIT(off)) != 0;
350 }

References GetBitvecSizeInBytes(), and m_buff_written.

Referenced by CountBlocksNotWrittenInRng(), GetLastDownloadedBlock(), GetNDownloadedBlocks(), and XrdPfc::File::Prefetch().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ UpdateDownloadCompleteStatus()

void XrdPfc::Info::UpdateDownloadCompleteStatus ( )
inline

Update complete status.

Definition at line 463 of file XrdPfcInfo.hh.

464 {
466  m_complete = (m_missingBlocks == 0);
467 }
int CountBlocksNotWrittenInRng(int firstIdx, int lastIdx) const
Check download status in given block range.
Definition: XrdPfcInfo.hh:452

References CountBlocksNotWrittenInRng(), m_bitvecSizeInBits, m_complete, and m_missingBlocks.

Referenced by Read().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ Write()

bool Info::Write ( XrdOssDF fp,
const char *  dname,
const char *  fname = 0 
)

Write number of blocks and read buffer size

Parameters
fpfile handle
dnamedirectory name for trace output
fnameoptional file name for trace output (can be included in dname)
Returns
true on success

Definition at line 268 of file XrdPfcInfo.cc.

269 {
270  TraceHeader trace_pfx("Write()", dname, fname);
271 
273  m_store.m_astatSize = (int32_t) m_astats.size();
274 
275  FpHelper w(fp, 0, m_trace, m_traceID, trace_pfx);
276 
277  if (w.Write(s_defaultVersion) ||
278  w.Write(m_store) ||
279  w.Write(CalcCksumStore()) ||
280  w.WriteRaw(m_buff_synced, GetBitvecSizeInBytes()) ||
281  w.WriteRaw(m_astats.data(), m_store.m_astatSize * sizeof(AStat)) ||
282  w.Write(CalcCksumSyncedAndAStats()))
283  {
284  return false;
285  }
286 
287  return true;
288 }
void CompactifyAccessRecords()
Compactify access records to the configured maximum.
Definition: XrdPfcInfo.cc:381

References CalcCksumStore(), CalcCksumSyncedAndAStats(), CompactifyAccessRecords(), GetBitvecSizeInBytes(), m_astats, XrdPfc::Info::Store::m_astatSize, m_buff_synced, m_store, m_trace, m_traceID, s_defaultVersion, and s_maxNumAccess.

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), XrdPfc::Cache::LocalFilePath(), and XrdPfc::File::Sync().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ WriteIOStat()

void Info::WriteIOStat ( Stats s)

Write bytes missed, hits, and disk.

Definition at line 431 of file XrdPfcInfo.cc.

432 {
433  m_astats.back().NumIos = s.m_NumIos;
434  m_astats.back().Duration = s.m_Duration;
435  m_astats.back().BytesHit = s.m_BytesHit;
436  m_astats.back().BytesMissed = s.m_BytesMissed;
437  m_astats.back().BytesBypassed = s.m_BytesBypassed;
438 }
long long m_BytesMissed
number of bytes served from remote and cached
Definition: XrdPfcStats.hh:40
long long m_BytesBypassed
number of bytes served directly through XrdCl
Definition: XrdPfcStats.hh:41
int m_Duration
total duration of all IOs attached
Definition: XrdPfcStats.hh:38
int m_NumIos
number of IO objects attached during this access
Definition: XrdPfcStats.hh:37
long long m_BytesHit
number of bytes served from disk
Definition: XrdPfcStats.hh:39

References m_astats, XrdPfc::Stats::m_BytesBypassed, XrdPfc::Stats::m_BytesHit, XrdPfc::Stats::m_BytesMissed, XrdPfc::Stats::m_Duration, and XrdPfc::Stats::m_NumIos.

Referenced by XrdPfc::File::Sync(), and WriteIOStatDetach().

+ Here is the caller graph for this function:

◆ WriteIOStatAttach()

void Info::WriteIOStatAttach ( )

Write open time in the last entry of access statistics.

Definition at line 422 of file XrdPfcInfo.cc.

423 {
425 
426  AStat as;
427  as.AttachTime = time(0);
428  m_astats.push_back(as);
429 }

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::Store::m_accessCnt, m_astats, and m_store.

◆ WriteIOStatDetach()

void Info::WriteIOStatDetach ( Stats s)

Write close time together with bytes missed, hits, and disk.

Definition at line 440 of file XrdPfcInfo.cc.

441 {
442  m_astats.back().DetachTime = time(0);
443  WriteIOStat(s);
444 }
void WriteIOStat(Stats &s)
Write bytes missed, hits, and disk.
Definition: XrdPfcInfo.cc:431

References m_astats, and WriteIOStat().

Referenced by XrdPfc::File::FinalizeSyncBeforeExit().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ WriteIOStatSingle() [1/2]

void Info::WriteIOStatSingle ( long long  bytes_disk)

Write single open/close time for given bytes read from disk.

Definition at line 446 of file XrdPfcInfo.cc.

447 {
449 
450  AStat as;
451  as.AttachTime = as.DetachTime = time(0);
452  as.NumIos = 1;
453  as.BytesHit = bytes_disk;
454  m_astats.push_back(as);
455 }

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::BytesHit, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::Store::m_accessCnt, m_astats, m_store, and XrdPfc::Info::AStat::NumIos.

Referenced by XrdPfc::Cache::ExecuteCommandUrl(), and XrdPfc::Cache::LocalFilePath().

+ Here is the caller graph for this function:

◆ WriteIOStatSingle() [2/2]

void Info::WriteIOStatSingle ( long long  bytes_disk,
time_t  att,
time_t  dtc 
)

Write open/close with given time and bytes read from disk.

Definition at line 457 of file XrdPfcInfo.cc.

458 {
460 
461  AStat as;
462  as.AttachTime = att;
463  as.DetachTime = dtc;
464  as.NumIos = 1;
465  as.Duration = dtc - att;
466  as.BytesHit = bytes_disk;
467  m_astats.push_back(as);
468 }

References XrdPfc::Info::AStat::AttachTime, XrdPfc::Info::AStat::BytesHit, XrdPfc::Info::AStat::DetachTime, XrdPfc::Info::AStat::Duration, XrdPfc::Info::Store::m_accessCnt, m_astats, m_store, and XrdPfc::Info::AStat::NumIos.

Member Data Documentation

◆ m_astats

◆ m_bitvecSizeInBits

int XrdPfc::Info::m_bitvecSizeInBits
protected

◆ m_buff_prefetch

unsigned char* XrdPfc::Info::m_buff_prefetch
protected

prefetch statistics

Definition at line 322 of file XrdPfcInfo.hh.

Referenced by ~Info(), ResizeBits(), SetBitPrefetch(), and TestBitPrefetch().

◆ m_buff_synced

unsigned char* XrdPfc::Info::m_buff_synced
protected

disk written state vector

Definition at line 320 of file XrdPfcInfo.hh.

Referenced by ~Info(), CalcCksumSyncedAndAStats(), Read(), ResizeBits(), SetAllBitsSynced(), SetBitSynced(), and Write().

◆ m_buff_written

unsigned char* XrdPfc::Info::m_buff_written
protected

download state vector

Definition at line 321 of file XrdPfcInfo.hh.

Referenced by ~Info(), Read(), ResizeBits(), SetBitWritten(), and TestBitWritten().

◆ m_complete

bool XrdPfc::Info::m_complete
protected

cached; if false, set to true when missingBlocks hit zero

Definition at line 328 of file XrdPfcInfo.hh.

Referenced by IsComplete(), ResizeBits(), SetAllBitsSynced(), SetBitWritten(), and UpdateDownloadCompleteStatus().

◆ m_hasPrefetchBuffer

bool XrdPfc::Info::m_hasPrefetchBuffer
protected

constains current prefetch score

Definition at line 329 of file XrdPfcInfo.hh.

Referenced by ResizeBits().

◆ m_missingBlocks

int XrdPfc::Info::m_missingBlocks
protected

cached, updated in SetBitWritten()

Definition at line 327 of file XrdPfcInfo.hh.

Referenced by ResizeBits(), SetBitWritten(), and UpdateDownloadCompleteStatus().

◆ m_store

◆ m_trace

XrdSysTrace* XrdPfc::Info::m_trace
protected

Definition at line 317 of file XrdPfcInfo.hh.

Referenced by GetTrace(), Read(), and Write().

◆ m_traceID

const char * Info::m_traceID = "CInfo"
static

Definition at line 308 of file XrdPfcInfo.hh.

Referenced by Read(), and Write().

◆ m_version

int XrdPfc::Info::m_version
protected

Definition at line 325 of file XrdPfcInfo.hh.

Referenced by GetVersion(), and Read().

◆ s_defaultVersion

const int Info::s_defaultVersion = 4
static

Definition at line 312 of file XrdPfcInfo.hh.

Referenced by Read(), and Write().

◆ s_infoExtension

◆ s_infoExtensionLen

const size_t Info::s_infoExtensionLen = strlen(Info::s_infoExtension)
static

◆ s_maxNumAccess

size_t Info::s_maxNumAccess = 20
static

Definition at line 311 of file XrdPfcInfo.hh.

Referenced by CompactifyAccessRecords(), XrdPfc::Cache::Config(), and Write().


The documentation for this class was generated from the following files: