Like any story and issue there are couple sides which are touching the subject. One is pulling strings to his side while the other to his.\u00a0They both think that they are right and in many cases they do. Specifically in Computer\u00a0Science there\u00a0is an option to find a solution that will satisfy couple sides of the picture without harming the other\u00a0while in the real and material world some things cannot be resolved in a similar way.<\/p>\n
Microsoft is a huge company which supplies Desktops, Servers, Mobile and other IT platforms solutions. Their services are global and else then the development of the software they provide security and enhancement updates\u00a0to their clients. Microsoft is committed to allow their clients fast and reliable updates submission\\transfer.<\/p>\n
Microsoft spreads updates for a very long time using the Internet but since\u00a0their software is complex \u00a0sometimes they are required to spread a huge(in size) and urgent updates.\u00a0Due to the fact that\u00a0the world of IT is as wide as the sea, Microsoft uses CDN suppliers these days to spread their software and updates to their clients. But it’s not enough since in local networks the download of updates from\u00a0a CDN over the Internet can be reduced into the local network using either a local centralized updates service or using some local Peer2Peer file transfer.<\/p>\n
In many networks of above 10 Desktops there is a local server which can be used to spread Microsoft Windows updates and still in many cases it’s the right solution and it’s a weird situation that many admins prefer to try other solutions rather that what they\u00a0were given to use.<\/p>\n
From a network admins perspective of things when Microsoft spreads a huge update to their client it can cause a Torrent(not the P2P one) of high network utilization. And since\u00a0the network lines needs to be available to a more urgent things which are mandatory . They need to somehow Police the traffic in a way that windows updates traffic\u00a0will not harm other clients.<\/p>\n
Small office networks that has 4 Desktops and 1 Server in most cases are not in real trouble unless\u00a0there is a use of some kind of Web or Cloud service which huge\u00a0Microsoft Updates will might cause issues with.<\/p>\n
For Small and Medium size offices with more then 20 Desktops the situation is that Microsoft Updates\u00a0repeated download can be prevented or managed in a way that will not cause issues to regular network traffic either by the local System Administrator or using Network level QOS. But when there is no System Administrator or there is some kind of combined job for couple areas, it can be easier to implement a more simple updates\u00a0solution in the “network” level rather then in a System level.<\/p>\n
From an ISP Network\u00a0Administrator point of view everything is bits. He needs to make sure that the “important” bits will get from\u00a0the client side of the network to the edge of it. Microsoft Updates can cause a headache of these are repeatedly downloaded over and over again by\u00a0each and every Microsoft client.\u00a0The fast solution to allow Microsoft Updates is to slow them down or host a local\u00a0Instance of Microsoft CDN partners.\u00a0While for Medium and above ISPs it would be simple to host a CDN instance or server, it would not be the same for small and big networks. Most big networks either do not care about these Updates since their lines are built to handle lots of traffic or they already have one of Microsoft CDN Instances already hosted in\u00a0their Racks.<\/p>\n
For the Small and Medium sized ISP’s the situation is a bit different since they can get “stuck”\u00a0when they have\u00a0what can be described as “Network Clients DOS”. The solution for them are mainly local caching or targeted traffic throttling. Any of these solution is not very simple and requires\u00a0knowledge in both Networking and the upper levels of the connection and only for the purpuse of debugging\u00a0some issues and deciding on the right approach.<\/p>\n
In the case of\u00a0SAT or long distance Wireless connections the admin\u00a0usually have limited resources which he can spare and Microsoft Updates\u00a0might not be as important as some\u00a0GPS based navigation software but they mostly downloaded “automatically” these days so it’s an obstacle and in these cases Microsoft Updates are being blocked in\u00a0couple layers from the IP to\u00a0the application.<\/p>\n
The issue is not related directly to Microsoft and their Updates\u00a0but indeed\u00a0they deserve respect since they will probably continue to be present in Space.\u00a0Since we are moving continuously towards the future we can assume that updates will be something important.\u00a0In the case of Distance Space Ships it is feasible \u00a0to assume that if multiple Ships will be out-there then a centralized\u00a0distribution point (like WSUS) will probably be used to distribute static content. When it is also possible that the maintenance of such a system will not be under a single layer administration due to\u00a0the complexity of the task. For these a “cache” or a “store” is the choice to distribute identical content to multiple Space Ships or Space Stations.<\/p>\n
When huge Microsoft Updates are being spread around the globe to their clients\u00a0Network and System Administrators reporting repeatedly \u00a0about higher consumption of bandwidth and other system resources.<\/p>\n
Since Microsoft is a “generic” product their updates are\u00a0created for many clients and not customized \u00a0for a specific one.\u00a0These are static objects\\content which being downloaded over and over again and again, due to this it’s possible to prevent repeated downloads of the content.<\/p>\n
Couple sides are\u00a0affected by the same issue.While Microsoft are off-loading or out-sourcing the distribution of the static content to other parties. And by that solving their responsibility to spread the updates and allowing their clients faster download of the updates, other parties on the network level are left to handle a weird situation. Some of these just want to “survive” a huge update while others wants to earn\u00a0couple more bucks to their monthly revenue. Some are more greedy while others are in-need.<\/p>\n
Microsoft do not offer ISP’s or network admins a caching solution for their update since this\u00a0is not their domain.\u00a0Microsoft leave the network admins the option to implement any solution they want but it requires an\u00a0expert that knows a thing about networking\u00a0HTTP and other areas which are the road and door\u00a0into the palace.<\/p>\n
In order to implement a caching solution for Microsoft Updates there is a need to know first\u00a0the structure of Microsoft Updates protocols and systems. There are not simple for all but there are couple issues inside this box.
\nThings to consider when handling the issue:<\/p>\n
We can try to approach the target from couple angles but first we need to know: what’s out there?
\nAlso note that these are not a junior System Administrator level task so talk\u00a0with others about the subject before diving to implement any solution.<\/p>\n
Microsoft offers a solution\u00a0to manage Windows Updates for Domains using a local service: WSUS<\/a> For small networks Squid can be used to Intercept \u00a0traffic of windows updates DATA channel over HTTP and to serve\u00a0static content locally. \u00a0Details can be found in the\u00a0wiki.squid-cache.org<\/a>\u00a0.<\/p>\n I have not implemented a caching solution using Nginx or Varnish for Windows Updates but it seems to me that it is possible and the proof for that is that we could cache YouTube static videos using these.<\/p>\n In Varnish VCL you can define each and every object cache key which\u00a0can somehow help when handling multiple domains for the same content.<\/p>\n In any case\u00a0that you will approach caching \u00a0you\u00a0will need to consider couple angles to the subject. Any solution have Pros and Cons if it’s OpenSource, Free or Paid.<\/p>\n Squid is a great piece of software that can help to cache static\u00a0content\u00a0and control web access. But like any\u00a0software\u00a0the project have limited resources and due to this and the complexity of the software you would find that caching\u00a0Windows Updates might not be as simple as it may seems like.<\/p>\n Like any cache solution squid uses the “pulling clients”(downloading) as a force of cache content population. And depends on the design of the cache other options might be better. But if a client will never\u00a0try to download the cache will never be populated. And\u00a0in some cached it is possible to predict what needs to be\u00a0pre-fetched into the cached and in these cases the clients might not be the right cache population force.<\/p>\n Squid handles range requests in a simple way and\u00a0if the full object wasn’t fully downloaded (ie the cache was not seeded with the object) the request will be served from the\u00a0remote origin service and\u00a0the client will not “drive” the cache to fully download the content object.<\/p>\n While it is possible to cause a full fetch based on a range request, currently squid will do that but\u00a0will force it on the whole service and not only on a specific domain or urls or responses.<\/p>\n A cache can be populated like WSUS does, it fetches the required objects and the spreads them to the local stations. And since a cache service is not a Microsoft legal client it will not be able to access some kind of “list”\u00a0of updates files\\objects which can be pre-fetched. It leaves the cache\u00a0no options to “predict” and pre-fetch the content but to wait for clients to first initiate contact with the origin service and only after that\u00a0was done to populate the cache.<\/p>\n Since the objects\u00a0from Microsoft Updates\u00a0might contain Private content such as a\u00a0registered bundled software the cache needs to differentiate between Private and Public objects. We can try to catch a bunch of domains using Varnish but since it was not\u00a0designed to operate this way there are lots of “breakable” parts in such a setup.<\/p>\n Nginx was designed to be a web server and a reverse proxy and white it can be used pretty easy to mirror windows updates in a similar way to what is mentioned in this article<\/a>\u00a0it’s not recommended without lots of experience and deep understanding of the subject.<\/p>\n Since Microsoft didn’t published yet any specification about\u00a0their updates infrastructure and\u00a0software it will always be considered a “hack” and due to this my recommendation is to have a “terminate” switch available at any point\u00a0to\u00a0avoid DOS from your side.<\/p>\n There are many security threats around caching so it is recommended to pull some security expert when implementing any solution. If you will do a bad job it is possible that your clients will get infected by malicious software and as a cache admin you will\u00a0need to pay\u00a0repair and data loss fees for your clients.<\/p>\n There are couple ways to “overflow” a cache solution and you will need to take good\u00a0care in the design of a cache system so it would not crash due to low resources or due to clients content demand.<\/p>\n Keep your cache software\u00a0safe from both local and remote\u00a0clients since they can hit your box pretty hard if\u00a0you will leave it vulnerable and open to the wide world.<\/p>\n The CDN networks are built to handle high load of traffic while a 7.2K RPMs HDD\u00a0maybe cannot compete a their SSD and the outcome would be slow download speed of updates.\u00a0So design your cache to be efficient\u00a0at-least compared to the origin service.<\/p>\n There are couple admins that will want their caching service to slow down the network demand but these are considered “harming”\u00a0for most networks but in some cases\u00a0it is the right solution when considering the benefit of critical systems compared to an update.<\/p>\n kb-3084568<\/a><\/p>\n Since Microsoft Updates are using an\u00a0encrypted and secure communication channel and another plain web(HTTP) based\u00a0transfer channel we can cache what’s in the plain public channel without touching any of the private data safely.<\/p>\n The solution will be composed of three parts:<\/p>\n We are aiming to statically store public content which can be stored on a SSD drive which will allow fast downloads speed.<\/p>\n In squid we will use a “store” cache_peer for all the domains including “.download.windowsupdate.com”. The store cache_peer proxy will “log” every request in a way that it can be re-used later. Every once in a while you will need to run the “fetcher” software that will use the logged requests in order to populate the\u00a0store objects.<\/p>\n This solution was designed for small networks but can be used with\u00a0multiple store servers and a big storage.\u00a0So test first to understand the situation with your network.<\/p>\n If you are a large ISP(thousands of clients) this solution was probably not designed for you but if you have enough on your shoulder you can test it and see how you might fit it into your organization.<\/p>\n Download the binaries from:\u00a0NgTech ms-updates-store The service and the fetcher script at: GIST<\/a><\/p>\n The binaries packs\u00a0contains static binaries for any OS I could create. Add into your squid.conf the next:<\/p>\n acl wu dstdom_regex download\\.windowsupdate\\.com$ If you have a specific volume that you want to store the\u00a0static updates in then change the location\u00a0“\/var\/storedata” both at the fetcher script and the systemd service file.<\/p>\n You can run the fetcher script under as a cron job according to your preference but it is recommended to\u00a0run it\u00a0in interval of at-least 6 hours.<\/p>\n Feel free to contact me directly via email or\u00a0other options.<\/p>\n Eliezer Croitoru
\nIt cannot be used in\u00a0an ISP or IX environment.<\/p>\nAn HTTP caching proxy such as Squid<\/h3>\n
A reverse proxy such as Nginx or Varnish<\/h3>\n
Pros And Cons for existing solutions<\/h2>\n
Squid<\/h3>\n
The Driving force of the cache<\/h4>\n
206 range requests<\/h4>\n
Prefetching<\/h4>\n
Public VS Private objects<\/h4>\n
\nIt is possible that if the cache will not honor objects privacy \u00a0a client will eventually fetch an object which was licensed to another organization\u00a0which is a series violation and exploitation of trust and credibility.<\/p>\nVarnish is a reverse proxy!<\/h3>\n
Nginx is a Web Server and a\u00a0Reverse Proxy, not a\u00a0Caching proxy.<\/h3>\n
The risks of any solution<\/h2>\n
Security<\/h3>\n
Cache overflows<\/h3>\n
Network attacks<\/h3>\n
Slow storage<\/h3>\n
Microsoft Updates infor for networks with a local\u00a0proxy<\/h2>\n
My solution – MS Updates\u00a0public-store<\/h2>\n
Overview<\/h3>\n
\n
\nEvery request will eventually end up with the outgoing IP of the special “store” cache_peer proxy so take it into consideration.<\/p>\nThe use cases\u00a0of the solution<\/h3>\n
Installation<\/h3>\n
\n<\/a>Download the systemd service file<\/a>.
\nDownload the fetcher bash script<\/a>.<\/p>\n
\nYou need to install them at \/usr\/bin\/ or at \/usr\/local\/x\/bin , choose the installation location of the binaries and change the location and file\u00a0names inside the scripts accordingly.<\/p>\n\n
\nacl wu dstdom_regex download\\.microsoft\\.com$
\nacl wu-rejects dstdom_regex\u00a0stats
\nacl GET method GET
\ncache_peer 127.0.0.1\u00a0parent 8080 0 proxy-only no-tproxy no-digest no-query no-netdb-exchange name=ms1
\ncache_peer_access ms1 allow GET\u00a0wu !wu-rejects
\ncache_peer_access ms1 deny all
\nnever_direct allow GET wu !wu-rejects
\nnever_direct deny all<\/p>\n<\/blockquote>\nThis is a beta version which was tested to work on a small production environment but needs to be verified to fit your use case. Please test first then sorry later.<\/h1>\n
\neliezer@ngtech.co.il<\/a>
\n