XRootD
XrdHttpReadRangeHandler Class Reference

#include <XrdHttpReadRangeHandler.hh>

+ Collaboration diagram for XrdHttpReadRangeHandler:

Classes

struct  Configuration
 
struct  Error
 
struct  UserRange
 

Public Types

typedef std::vector< UserRangeUserRangeList
 

Public Member Functions

 XrdHttpReadRangeHandler (const Configuration &conf)
 
const ErrorgetError () const
 return the Error object More...
 
bool isFullFile ()
 indicates when there were no valid Range head ranges supplied More...
 
bool isSingleRange ()
 indicates a single range (implied whole file, or single range) or empty file More...
 
const UserRangeListListResolvedRanges ()
 return resolved (i.e. obsolute start and end) byte ranges desired More...
 
const XrdHttpIOListNextReadList ()
 return XrdHttpIOList for sending to read or readv More...
 
void NotifyError ()
 Force handler to enter error state. More...
 
int NotifyReadResult (const ssize_t ret, const UserRange **const urp, bool &start, bool &allend)
 Advance internal counters concerning received bytes. More...
 
void ParseContentRange (const char *const line)
 parse the line after a "Range: " http request header More...
 
void reset ()
 resets this handler More...
 
int SetFilesize (const off_t sz)
 sets the filesize, used during resolving and issuing range requests More...
 

Static Public Member Functions

static int Configure (XrdSysError &Eroute, const char *const parms, Configuration &cfg)
 

Static Public Attributes

static constexpr size_t READV_MAXCHUNKS = 512
 
static constexpr size_t READV_MAXCHUNKSIZE = 512*1024
 
static constexpr size_t RREQ_MAXSIZE = 8*1024*1024
 

Detailed Description

Class responsible for parsing the HTTP Content-Range header coming from the client, generating appropriate read ranges for read or readv and tracking the responses to the requests.

Definition at line 36 of file XrdHttpReadRangeHandler.hh.

Member Typedef Documentation

◆ UserRangeList

Definition at line 103 of file XrdHttpReadRangeHandler.hh.

Constructor & Destructor Documentation

◆ XrdHttpReadRangeHandler()

XrdHttpReadRangeHandler::XrdHttpReadRangeHandler ( const Configuration conf)
inline

Constructor. Supplied with an Configuration object. The supplied object remains owned by the caller, but should remain valid throughout the lifetime of the ReadRangeHandler.

Parameters

Definition at line 113 of file XrdHttpReadRangeHandler.hh.

114  {
115  rRequestMaxBytes_ = RREQ_MAXSIZE;
116  vectorReadMaxChunkSize_ = READV_MAXCHUNKSIZE;
117  vectorReadMaxChunks_ = READV_MAXCHUNKS;
118 
119  if( conf.haveSizes )
120  {
121  vectorReadMaxChunkSize_ = conf.readv_ior_max;
122  vectorReadMaxChunks_ = conf.readv_iov_max;
123  rRequestMaxBytes_ = conf.reqs_max;
124  }
125  reset();
126  }
static constexpr size_t READV_MAXCHUNKSIZE
void reset()
resets this handler
static constexpr size_t RREQ_MAXSIZE
static constexpr size_t READV_MAXCHUNKS

References XrdHttpReadRangeHandler::Configuration::haveSizes, XrdHttpReadRangeHandler::Configuration::readv_ior_max, XrdHttpReadRangeHandler::Configuration::readv_iov_max, READV_MAXCHUNKS, READV_MAXCHUNKSIZE, XrdHttpReadRangeHandler::Configuration::reqs_max, reset(), and RREQ_MAXSIZE.

+ Here is the call graph for this function:

Member Function Documentation

◆ Configure()

int XrdHttpReadRangeHandler::Configure ( XrdSysError Eroute,
const char *const  parms,
Configuration cfg 
)
static

Parses a configuration into a Configuration object.

Parameters

Definition at line 42 of file XrdHttpReadRangeHandler.cc.

47 {
48  if( !parms ) return 0;
49 
50  std::vector<std::string> splitArgs;
51  XrdOucTUtils::splitString( splitArgs, parms, "," );
52  if( splitArgs.size() < 2 ) return 0;
53 
54  //----------------------------------------------------------------------------
55  // params is expected to be "<readv_ior_max>,<readv_iov_max>"
56  //----------------------------------------------------------------------------
57  std::string iorstr = splitArgs[0];
58  std::string iovstr = splitArgs[1];
59  XrdOucUtils::trim( iorstr );
60  XrdOucUtils::trim( iovstr );
61 
62  int val;
63  if( XrdOuca2x::a2i( Eroute, "Error reading specific value of readv_ior_max",
64  iorstr.c_str(), &val, 1, -1 ) )
65  {
66  return -1;
67  }
68 
69  cfg.readv_ior_max = val;
70  if( XrdOuca2x::a2i( Eroute, "Error reading specific value of readv_iov_max",
71  iovstr.c_str(), &val, 1, -1 ) )
72  {
73  return -1;
74  }
75 
76  cfg.readv_iov_max = val;
77  cfg.reqs_max = RREQ_MAXSIZE;
78  cfg.haveSizes = true;
79 
80  return 0;
81 }
static void splitString(Container &result, const std::string &input, const std::string &delimiter)
Split a string.
Definition: XrdOucTUtils.hh:51
static void trim(std::string &str)
static int a2i(XrdSysError &, const char *emsg, const char *item, int *val, int minv=-1, int maxv=-1)
Definition: XrdOuca2x.cc:45

References XrdOuca2x::a2i(), XrdHttpReadRangeHandler::Configuration::haveSizes, XrdHttpReadRangeHandler::Configuration::readv_ior_max, XrdHttpReadRangeHandler::Configuration::readv_iov_max, XrdHttpReadRangeHandler::Configuration::reqs_max, RREQ_MAXSIZE, XrdOucTUtils::splitString(), and XrdOucUtils::trim().

+ Here is the call graph for this function:

◆ getError()

const XrdHttpReadRangeHandler::Error & XrdHttpReadRangeHandler::getError ( ) const

return the Error object

getter for the Error object. The object can be inspected with its operator bool() method to indicate an error has happened. Error code and message are available in other members of Error.

Definition at line 86 of file XrdHttpReadRangeHandler.cc.

87 {
88  return error_;
89 }

◆ isFullFile()

bool XrdHttpReadRangeHandler::isFullFile ( )

indicates when there were no valid Range head ranges supplied

Indicates no valid Range header was given and thus the implication is that whole file is required. A range or ranges may be given that cover the whole file but that situation is not detected.

Definition at line 94 of file XrdHttpReadRangeHandler.cc.

95 {
96  return rawUserRanges_.empty();
97 }

◆ isSingleRange()

bool XrdHttpReadRangeHandler::isSingleRange ( )

indicates a single range (implied whole file, or single range) or empty file

Incidcates whether there is a single range, either given by a Range header with single range or implied by having no Range header. Also returns true for an empty file, although there is no range of bytes.

Returns
true if there is a single range.

Definition at line 102 of file XrdHttpReadRangeHandler.cc.

103 {
104  if( !rangesResolved_ )
105  resolveRanges();
106 
107  return( resolvedUserRanges_.size() <= 1 );
108 }

Referenced by XrdHttpReq::ProcessHTTPReq().

+ Here is the caller graph for this function:

◆ ListResolvedRanges()

const XrdHttpReadRangeHandler::UserRangeList & XrdHttpReadRangeHandler::ListResolvedRanges ( )

return resolved (i.e. obsolute start and end) byte ranges desired

Returns a reference of the list of ranges. These are resolved, meaning that if there was no Range header, or it was in the form -N or N-, the file size is used to compute the actual range of bytes that are needed. The list remains owned by the handler and may be invalidated on reset().

Returns
List of ranges in a UserRangeList object. The returned list may be empty, i.e. for an empty file or if there is an error. Use getError() to see if there is an error.

Definition at line 113 of file XrdHttpReadRangeHandler.cc.

114 {
115  static const UserRangeList emptyList;
116 
117  if( !rangesResolved_ )
118  resolveRanges();
119 
120  if( error_ )
121  return emptyList;
122 
123  return resolvedUserRanges_;
124 }
std::vector< UserRange > UserRangeList

◆ NextReadList()

const XrdHttpIOList & XrdHttpReadRangeHandler::NextReadList ( )

return XrdHttpIOList for sending to read or readv

Requests a XrdHttpIOList (vector of XrdOucIOVec2) that describes the next bytes that need to be fetched from a file. If there is more than one chunk it is size appropriately for a readv request, if there is one request it should be sent as a read request. Therefore the chunks do not necessarily correspond to the ranges the user requested. The caller issue the requests in the order provided and call NotifyReadResult with the ordered results.

Returns
a reference to a XrdHttpIOList. The object remains owned by the handler. It may be invalided by a new call to NextReadList() or reset(). The returned list may be empty, which implies no more reads are needed One can use getError() to see if there is an error.

Definition at line 129 of file XrdHttpReadRangeHandler.cc.

130 {
131  static const XrdHttpIOList emptyList;
132 
133  if( !rangesResolved_ )
134  resolveRanges();
135 
136  if( error_ )
137  return emptyList;
138 
139  if( !splitRange_.empty() )
140  {
141  if( currSplitRangeIdx_ == 0 && currSplitRangeOff_ == 0 )
142  {
143  //------------------------------------------------------------------------
144  // Nothing read: Prevent scenario where data is expected but none is
145  // actually read E.g. Accessing files which return the results of a script
146  //------------------------------------------------------------------------
147  error_.set( 500, "Stopping request because more data is expected "
148  "but no data has been read." );
149  return emptyList;
150  }
151 
152  //--------------------------------------------------------------------------
153  // we may have some unacknowledged portion of the last range; maybe due to a
154  // short read. so remove what was received and potentially reissue.
155  //--------------------------------------------------------------------------
156 
157  trimSplit();
158  if( !splitRange_.empty() )
159  return splitRange_;
160  }
161 
162  if( splitRangeIdx_ >= resolvedUserRanges_.size() )
163  return emptyList;
164 
165  splitRanges();
166 
167  return splitRange_;
168 }
std::vector< XrdOucIOVec2 > XrdHttpIOList
Definition: XrdHttpUtils.hh:95
void set(int rc, const std::string &m)

References XrdHttpReadRangeHandler::Error::set().

Referenced by XrdHttpReq::ProcessHTTPReq().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ NotifyError()

void XrdHttpReadRangeHandler::NotifyError ( )

Force handler to enter error state.

Force the handler to enter error state. Sets a generic error message if there was not already an error.

Definition at line 173 of file XrdHttpReadRangeHandler.cc.

174 {
175  if( error_ )
176  return;
177 
178  error_.set( 500, "An error occured." );
179 }

References XrdHttpReadRangeHandler::Error::set().

Referenced by XrdHttpReq::File().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ NotifyReadResult()

int XrdHttpReadRangeHandler::NotifyReadResult ( const ssize_t  ret,
const UserRange **const  urp,
bool &  start,
bool &  allend 
)

Advance internal counters concerning received bytes.

Notifies the handler about the arrival of bytes from a read or readv request. The handler tracks the progress of the arriving bytes against the bytes ranges the user requested.

Parameters
retthe number of bytes received
urpa pointer to a pointer of a UserRange object. If urp is not nullptr, the pointer to a UserRange is returned that describes the current range associated with the received bytes. The handler retains ownership of the returned object. reset() of the handler invalidates the UserRange object.
startis an output bool parameter that indicates whether the received bytes mark the start of a UserRange.
allendis an output bool parameter that indicates whether the received bytes mark the end of all the UserRanges
Returns
0 upon success, -1 if an error happened. One needs to call the getError() method to return the error.

Definition at line 184 of file XrdHttpReadRangeHandler.cc.

191 {
192  if( error_ )
193  return -1;
194 
195  if( ret == 0 )
196  return 0;
197 
198  if( ret < 0 )
199  {
200  error_.set( 500, "Range handler read failure." );
201  return -1;
202  }
203 
204  if( !rangesResolved_ )
205  {
206  error_.set( 500, "Range handler ranges not yet resolved." );
207  return -1;
208  }
209 
210  if( splitRange_.empty() )
211  {
212  error_.set( 500, "No ranges being read." );
213  return -1;
214  }
215 
216  start = false;
217  allend = false;
218 
219  if( currSplitRangeIdx_ >= splitRange_.size() ||
220  resolvedRangeIdx_ >= resolvedUserRanges_.size() )
221  {
222  error_.set( 500, "Range handler index invalid." );
223  return -1;
224  }
225 
226  if( urp )
227  *urp = &resolvedUserRanges_[resolvedRangeIdx_];
228 
229  if( resolvedRangeOff_ == 0 )
230  start = true;
231 
232  const int clen = splitRange_[currSplitRangeIdx_].size;
233 
234  const off_t ulen = resolvedUserRanges_[resolvedRangeIdx_].end
235  - resolvedUserRanges_[resolvedRangeIdx_].start + 1;
236 
237  currSplitRangeOff_ += ret;
238  resolvedRangeOff_ += ret;
239 
240  if( currSplitRangeOff_ > clen || resolvedRangeOff_ > ulen )
241  {
242  error_.set( 500, "Range handler read crossing chunk boundary." );
243  return -1;
244  }
245 
246  if( currSplitRangeOff_ == clen )
247  {
248  currSplitRangeOff_ = 0;
249  currSplitRangeIdx_++;
250 
251  if( currSplitRangeIdx_ >= splitRange_.size() )
252  {
253  currSplitRangeIdx_ = 0;
254  splitRange_.clear();
255  }
256  }
257 
258  if( resolvedRangeOff_ == ulen )
259  {
260  resolvedRangeIdx_++;
261  resolvedRangeOff_ = 0;
262  if( resolvedRangeIdx_ >= resolvedUserRanges_.size() )
263  allend = true;
264  }
265 
266  return 0;
267 }

References XrdHttpReadRangeHandler::Error::set().

Referenced by XrdHttpReq::File().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ ParseContentRange()

void XrdHttpReadRangeHandler::ParseContentRange ( const char *const  line)

parse the line after a "Range: " http request header

Parses the Content-Range header value and sets the ranges within the object.

Parameters
linethe line under the format "bytes=0-19, 25-30" In case the parsing fails any partial results are cleared. There is no error notification as the rest of the request processing should continue in any case.

Definition at line 272 of file XrdHttpReadRangeHandler.cc.

273 {
274  char *str1, *saveptr1, *token;
275 
276  std::unique_ptr< char, decltype(std::free)* >
277  line_copy { strdup( line ), std::free };
278 
279  //----------------------------------------------------------------------------
280  // line_copy is argument of the Range header.
281  //
282  // e.g. "bytes=15-17,20-25"
283  // We skip the unit prefix (upto first '='). We don't
284  // enforce this prefix nor check what it is (e.g. 'bytes')
285  //----------------------------------------------------------------------------
286 
287  str1 = line_copy.get();
288  token = strchr(str1,'=');
289  if (token) str1 = token + 1;
290 
291  //----------------------------------------------------------------------------
292  // break up the ranges and process each
293  //----------------------------------------------------------------------------
294 
295  for( ; ; str1 = NULL )
296  {
297  token = strtok_r( str1, " ,\n\r", &saveptr1 );
298  if( token == NULL )
299  break;
300 
301  if( !strlen(token) ) continue;
302 
303  const int rc = parseOneRange( token );
304  if( rc )
305  {
306  //------------------------------------------------------------------------
307  // on error we ignore the whole range header
308  //------------------------------------------------------------------------
309  rawUserRanges_.clear();
310  return;
311  }
312  }
313 }

Referenced by XrdHttpReq::parseLine().

+ Here is the caller graph for this function:

◆ reset()

void XrdHttpReadRangeHandler::reset ( )

resets this handler

Resets the object state, ready for handling a new request.

Definition at line 318 of file XrdHttpReadRangeHandler.cc.

319 {
320  error_.reset();
321  rawUserRanges_.clear();
322  rawUserRanges_.shrink_to_fit();
323  resolvedUserRanges_.clear();
324  resolvedUserRanges_.shrink_to_fit();
325  splitRange_.clear();
326  splitRange_.shrink_to_fit();
327  rangesResolved_ = false;
328  splitRangeIdx_ = 0;
329  splitRangeOff_ = 0;
330  currSplitRangeIdx_ = 0;
331  currSplitRangeOff_ = 0;
332  resolvedRangeIdx_ = 0;
333  resolvedRangeOff_ = 0;
334  filesize_ = 0;
335 }

References XrdHttpReadRangeHandler::Error::reset().

Referenced by XrdHttpReadRangeHandler(), and XrdHttpReq::reset().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ SetFilesize()

int XrdHttpReadRangeHandler::SetFilesize ( const off_t  sz)

sets the filesize, used during resolving and issuing range requests

Notifies of the current file size. This information is required for processing range requests that imply reading to the end or a certain position before the end of a file. It is also used to determine when read or readv need no longer be issued when reading the whole file. Can be called once or more, after reset() but before isSingleRange(), ListResolvedRanges() or NextReadList() methods.

Parameters
szthe size of the file
Returns
0 upon success, -1 if an error happened. One needs to call the getError() method to return the error.

Definition at line 340 of file XrdHttpReadRangeHandler.cc.

341 {
342  if( error_ )
343  return -1;
344 
345  if( rangesResolved_ )
346  {
347  error_.set( 500, "Filesize notified after ranges resolved." );
348  return -1;
349  }
350 
351  filesize_ = fs;
352  return 0;
353 }

References XrdHttpReadRangeHandler::Error::set().

+ Here is the call graph for this function:

Member Data Documentation

◆ READV_MAXCHUNKS

constexpr size_t XrdHttpReadRangeHandler::READV_MAXCHUNKS = 512
staticconstexpr

These are defaults for: READV_MAXCHUNKS Max length of the XrdHttpIOList vector. READV_MAXCHUNKSIZE Max length of a XrdOucIOVec2 element. RREQ_MAXSIZE Max bytes to issue in a whole readv/read.

Definition at line 45 of file XrdHttpReadRangeHandler.hh.

Referenced by XrdHttpReadRangeHandler().

◆ READV_MAXCHUNKSIZE

constexpr size_t XrdHttpReadRangeHandler::READV_MAXCHUNKSIZE = 512*1024
staticconstexpr

Definition at line 46 of file XrdHttpReadRangeHandler.hh.

Referenced by XrdHttpReadRangeHandler().

◆ RREQ_MAXSIZE

constexpr size_t XrdHttpReadRangeHandler::RREQ_MAXSIZE = 8*1024*1024
staticconstexpr

Definition at line 47 of file XrdHttpReadRangeHandler.hh.

Referenced by XrdHttpReadRangeHandler(), and Configure().


The documentation for this class was generated from the following files: