Using MongoDB in Azure DocumentDB with .NET driver throws MongoCommandException

Using MongoDB in Azure DocumentDB with .NET driver throws MongoCommandException

I am using Azure DocumentDB to manipulate a MongoDB collection.
I have a 1000 RU/s limit according to my Azure Standard plan.
The issue I experience occurs when I try filtering and sorting some data from my MongoDB collection.
Here is my C# code using .NET MongoDB driver 2.4.2.0:
// GET api/movies
[HttpGet]
public async Task Get([RequiredFromQuery] int page, [FromQuery] int limit,
[FromQuery] string quality, [FromQuery] int minimumRating, [FromQuery] string queryTerm, [FromQuery] string genre, [FromQuery] string sortBy, [FromQuery] string orderBy)
{
var nbMoviesPerPage = 20;
if (limit >= 20 && limit <= 50) { nbMoviesPerPage = limit; } var currentPage = 1; if (page >= 1)
{
currentPage = page;
}

var movies = _mongoDbService.GetCollection(Constants.MoviesCollectionName);
var filter = Builders.Filter.Empty;
var sort = Builders.Sort.Descending(movie => movie.DateUploadedUnix);

if (!string.IsNullOrWhiteSpace(quality) &&
(quality == “720p” || quality == “1080p” || quality.ToLower() == “3d”))
{
filter = filter & Builders.Filter.Eq(“torrents.quality”, quality);
}

if (minimumRating > 0 && minimumRating < 10) { filter = filter & Builders.Filter.Gt(“rating”, minimumRating);
}

if (!string.IsNullOrWhiteSpace(queryTerm))
{
filter = filter &
(Builders.Filter.Regex(“imdb_code”, new BsonRegularExpression(“/^” + queryTerm + “$/i”)) |
Builders.Filter.Regex(“title”, new BsonRegularExpression(“/^” + queryTerm + “$/i”)) |
Builders.Filter.Regex(“cast.name”, new BsonRegularExpression(“/^” + queryTerm + “$/i”)) |
Builders.Filter.Regex(“cast.imdb_code”, new BsonRegularExpression(“/^” + queryTerm + “$/i”)));
}

if (!string.IsNullOrWhiteSpace(genre))
{
filter = filter & Builders.Filter.In(“genres”, new List
{
genre
});
}

if (!string.IsNullOrWhiteSpace(sortBy))
{
switch (sortBy)
{
case “title”:
sort = Builders.Sort.Ascending(movie => movie.Title);
break;
case “year”:
sort = Builders.Sort.Descending(movie => movie.Year);
break;
case “rating”:
sort = Builders.Sort.Descending(movie => movie.Rating);
break;
case “peers”:
sort =
Builders.Sort.Descending(movie => movie.Torrents.Select(torrent => torrent.Peers));
break;
case “seeds”:
sort =
Builders.Sort.Descending(movie => movie.Torrents.Select(torrent => torrent.Seeds));
break;
case “download_count”:
sort = Builders.Sort.Descending(movie => movie.DownloadCount);
break;
case “like_count”:
sort = Builders.Sort.Descending(movie => movie.LikeCount);
break;
case “date_added”:
sort = Builders.Sort.Descending(movie => movie.DateUploadedUnix);
break;
default:
sort = Builders.Sort.Descending(movie => movie.DateUploadedUnix);
break;
}
}

var filteredMovies = movies.Find(filter);
var totalTask = filteredMovies.CountAsync();
var moviesTask = filteredMovies.Skip((currentPage – 1) * nbMoviesPerPage).Limit(nbMoviesPerPage).Sort(sort).ToListAsync();
await Task.WhenAll(totalTask, moviesTask);

if (!string.IsNullOrWhiteSpace(orderBy))
{
switch (orderBy)
{
case “desc”:
moviesTask.Result.Reverse();
break;
}
}

return
Json(new MovieResponse
{
TotalMovies = totalTask.Result,
Movies = JsonConvert.DeserializeObject>(moviesTask.Result.ToJson())
});
}

When I call my controller with the route api/movies?page=1&queryTerm=TitaniC, the MongoDB command output is :
{find({ “$or” : [{ “imdb_code” : /^TitaniC$/i }, { “title” : /^TitaniC$/i }, { “cast.name” : /^TitaniC$/i }, { “cast.imdb_code” : /^TitaniC$/i }] }).sort({ “date_uploaded_unix” : -1 }).skip(0).limit(20)}
which seems fine and retrieves the appropriate documents.
But quite randomly, I get a 429 HTTP error from Azure telling me:
‘Command find failed: Message: {“Errors”:[“Request rate is large”]}
However, I call my API once and there is no other request…

Solutions/Answers:

Solution 1:

Why this happens;- when MongoDB fails to use an index obtain the sort order for a query, then it sorts the results in memory. If the sort operation consumes more than 32 megabytes, the above error is then thrown. One of the possible ways to solve this problem is using the allowDiskUse option to enable aggregation pipeline stages to write data to temporary files. Checkout My blog for details

var data = await movies.Aggregate(new AggregateOptions
        {
            AllowDiskUse = true,
        })
            .Match({})//Insert your query here
            .Skip(2)
            .Sort({ "_id", -1 } )//Insert your sort options here
            .Limit(20)
            .ToListAsync();

References