HttpClient计算多个URL远程文件大小

mark12345

我想计算1000个以上网址的总文件大小。任务运行时,我想显示当前数量和总数。但是,如果我添加您可以在代码中看到的信息,则会收到错误ssl消息。

这是我得到的错误:

    "Unhandled exception. System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.IO.IOException:  Received an unexpected EOF or 0 bytes from the transport stream.
   at System.Net.Security.SslStream.ReceiveBlobAsync[TIOAdapter](TIOAdapter adapter)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)"

您能否建议显示计数的最佳方法是什么?

 #region GET REMOTE FILE SIZE
public static async Task<string> GetRemoteFileSize(this List<string> urls) {
    long sum = 0;
    long localSum = 0;
    var tasks = new List<Task<long?>>();
    for(int i = 0; i < urls.Count; i++) {
        var url = urls[i];
        tasks.Add(GetTotalBytes(url));
        //if i will add here the counts, it's not getting the actual count
        Console.Title = $"{i} of {urls.Count}";
    }
    
    for (int i = 0; i < tasks.Count; i++) {
        Task<long?> task = tasks[i];
        localSum += Convert.ToInt32(await task.ConfigureAwait(false));
        Console.Title = $"{i} of {urls.Count}"; //if I will add this, I'm getting the SSL error even if I already added this " ServerCertificateCustomValidationCallback = (sender, cert, chain, sslPolicyErrors) => { return true; }," to my httpclient handler
    }
    await Task.WhenAll(tasks.ToArray()).ConfigureAwait(false);
    return Interlocked.Add(ref sum, localSum).FormatFileSize();
}
private static async Task<long?> GetTotalBytes(string url) {

    Uri uri = new Uri(url, UriKind.Absolute);
    uri.SetServiceManagerConnection();

    using var request = new HttpRequestMessage(HttpMethod.Head, uri);
    request.AddAllCommonHeaders(uri);
    using var response = await Client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false);
    HttpStatusCode statusCode = response.StatusCode;

    if (!response.IsSuccessStatusCode) {
        throw new InvalidDataException($"Error occured: {statusCode} {(int)statusCode}");
    }

    return response.Content.Headers.ContentLength.GetValueOrDefault();
}
#endregion GET REMOTE FILE SIZE

//我使用的课程

static string FormatBytes(this long bytes) {
        var unit = 1024;
        if (bytes < unit) { return $"{bytes} B"; }
        var exp = (int)(Math.Log(bytes) / Math.Log(unit));
        return $"{bytes / Math.Pow(unit, exp):F2} {("KMGTPE")[exp - 1]}B";
    }
    public static string FormatFileSize(this long bytes) => bytes.FormatBytes();

 public static void SetServiceManagerConnection(this Uri uri) {
        var sp = ServicePointManager.FindServicePoint(uri);
        sp.ConnectionLimit = 20; //default 2 //The number of connections per end point.
        sp.UseNagleAlgorithm = false; //Nagle’s algorithm is a means of improving the efficiency of TCP/IP networks by reducing the number of packets that need to be sent over the network
        sp.Expect100Continue = false; //save bandwidth before sending huge object for post and put request to ensure remote end point is up and running.
        sp.ConnectionLeaseTimeout = (int)TimeSpan.FromMinutes(1).TotalMilliseconds;//60 * 1000; // 1 minute
    }

更新:这是可行的:

  static HttpClient Client { get; set; } = new(new HttpClientHandler { 
        Proxy = null, 
        UseProxy = false,
        SslProtocols = (SslProtocols)(SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12 | SecurityProtocolType.Tls13)
    });
eriksv88

已更新: HTTPS出现TLS握手错误时,通常会出现此错误。您应该指定SecurityProtocol

System.Net.ServicePointManager.SecurityProtocol = 
SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12 | SecurityProtocolType.Tls13;

参考:https : //docs.microsoft.com/zh-cn/dotnet/api/system.net.securityprotocoltype

本文收集自互联网,转载请注明来源。

如有侵权,请联系[email protected] 删除。

编辑于
0

我来说两句

0条评论
登录后参与评论

相关文章

来自分类Dev

获取远程文件的大小

来自分类Dev

scp将多个文件从远程计算机复制到本地计算机

来自分类Dev

从终端获取远程文件的大小

来自分类Dev

计算平均文件大小

来自分类Dev

如何计算SQL Server 2008中位于不同计算机(远程)上的文件夹的大小?

来自分类Dev

在UNIX中仅从远程计算机复制文件详细信息(文件名,大小,时间)

来自分类Dev

获取远程URL的文件大小,而无需在Google App Engine(php)中下载

来自分类Dev

在SQL中计算文件大小

来自分类Dev

如何计算protobuf文件大小?

来自分类Dev

从HTTPS网址获取远程文件大小

来自分类Dev

PHP检查远程文件夹的大小

来自分类Dev

删除远程计算机上的文件

来自分类Dev

JavaScript验证多个文件的大小

来自分类Dev

在远程计算机上查找RAM的大小

来自分类Dev

计算多个文件中行的出现

来自分类Dev

限制跨多个文件的文件上传大小

来自分类Dev

PHP读取内容后计算文件大小

来自分类Dev

如何计算位图文件的大小?

来自分类Dev

du计算文件/目录的数量而不是大小

来自分类Dev

如何计算目录中文件的整体大小

来自分类Dev

计算 BeFS 支持的最大文件大小

来自分类Dev

PHP从远程URL获取XML不是文件

来自分类Dev

PHPMailer-使用URL附加远程文件

来自分类Dev

PHP从远程URL获取XML不是文件

来自分类Dev

通过Ruby中的提取链接检查远程文件大小

来自分类Dev

如果文件大于给定大小,则阻止从远程源加载

来自分类Dev

有什么方法可以不下载而获取远程文件的大小?

来自分类Dev

如果文件大于给定大小,则阻止从远程源加载

来自分类Dev

HttpClient-下载之前下载文件的大小

Related 相关文章

热门标签

归档