Category Archives: squid-3.5

squid-3.5, squid-4.0, squid-release, SquidBlocker

How accurate Statistics are? The SquidBlocker Way.

How accurate Statistics are?
The SquidBlocker Way.

SQUID 3.5.26 + 4.0.20 RELEASED

I have been working with psychologists and psychiatrists in the last ten plus years and an idea that emerged from all these sessions:

“Computer Assisted Analysis”

The basic idea was a bit raw and wasn’t mature enough to my taste. It was composed of a mathematical function that always results in the right answer\result whether the patient was mentally ill in the spectrums of Schizophrenia to Bipolar Disorder or “Normal”.

The idea could be very nice as a solution for a Mathematical Problem resolution but wasn’t enough to decide a human life fate. There is a need for a much more robust idea\concept then the non-mature one. One of the big issues with their basic concept was that I saw the mathematical equation that the MD’s were using to describe the concept and I saw how simple and unrealistic the it was. If the way to find out if a human is mentally ill is so simple then the human mind is much lower in it’s complexity compared to a flower or piece of grass or paper. As soon as I started proving the ridiculousness for the idea it was taken off the board. I was so happy at the time that so many lives were spared from a false positive match for a Schizophrenia and Bipolar Disorder.

But it’s just not enough to state “it’s not good” there is a need for an alternative! If you claim that something is missing couple things you must understand that some alternative should be offered, and if not a full mature alternative then a simple enough new direction should be the basic alternative.

The professional Therapists, Psychologists and Psychiatrists I was working with was thriving to get an idea which will help them analyze and treat these patients. And while it’s nice to use a brand new solution or formula that will assist them I believed that there are existing concepts and tools which can help us to grasp the patient state without re-inventing the wheel.

The basic idea is to find the right tool that will help us analyze the patient in a way that he will want to participate in the therapy process. The conflict which the therapist\psychiatrists  is standing in-front of is to choose the right “poison” that will help the specific patient. For example a Lithium consumer can be spotted in a second by a smart\experienced enough person. And the MD explained to me: “Do you see the color of his skin on the face? this is closer to white then to the red of flesh.”(this is one of the ways to see it)

I was amazed by what can the human body reflect to others. And then the decision became much more complex since we need to both protect the patient from revealing his “secret” of being a psychiatrist patient and also to try and find the right analysis let alone treat him. There are some “easy” cases which the basic concept of analysis will catch but most cases are much more complex to analyze find and treat. Also once you have analyzed the patient it’s pretty hard to change this “verdict”. In a sense it’s actually a matter of life and death when looking on how the verdict will affect the patient therapy, treatment and life.

At that stage the group of therapists revealed to me that they had failures while treating patients and for some reason there where patients which choose to take the path of “Another Life” by killing them self in some very artistic ways. A thing which the modern world accepts as a failure of treatment compared to the Japanese old tradition that defines such an act as an honorable and noble one.

I admit that while working some basic concept of analytical model I have seen in real life how a kid tries to take his life using a sharp razor blade, yes the one for the hairs in the legs or face. At the time I understood how much we need to be careful with blades!!!

In my culture there is a custom to not pass or use a blade on specific areas of the body what so ever. I was wondering for ages about at least one reason for why we were ordered to not use a blade on specific areas of the body. The answer was reflected to me from reality, from this kid that was trying to use the blade to take\end his life with this simple tool. This simple tool was in use in his house by his parent and was so “daily” that it was simple for the kid to think about using it for whatever he was thinking about.

This lecture taught me about the power of giving “Example” by our-self to others. We were given the gift of life and for a very long time I wanted to build a tool that will help others and specifically to these who are in distress, man’s or woman’s !

  • So what is the right thing to do?
  • What is the right tool to use?
  • What would help others in the most optimal way?

In order to get to the right answer I had to learn couple new sides of science in life since I was young and didn’t had enough experience compared to the MD’s I was working with on the project.

There is a known comparison between a CPU to a human thinking or mind which led me to a new kind of expression. We can use the Computer Science world and terms to express ideas the same way that a mathematical equation can express ideas and concepts.

Using this analogy\methodology of CS I tried to categorize CPU, DISK and RAM as a model of the components in the human mind and body.

As a therapist told me: We need to evaluate these three basic components and couple others like the above stack to try and understand the patient mind. Some CPU’s need a stress test while other’s DISK or RAM, depends on the patient. I decided to try and write a concept (in the form of a software) which will try to use the lowest level of resources in the process. Saying this would also mean for both my budget aspect of things and the expense which the client\patient needs to invest on it.

I will jump couple steps ahead since until now it’s kind of an “Assembly” level of things and will move forward to the actual result.

SquidBlocker!
Thats one of my answers.

I took the time learning the “Squid” way of things while comparing it to Varnish, Nginx and many other web and web caching services. They all seems to have a special way of expressing what’s good should look like. Eventually I decided that I should not invent the wheel but learn from the best and based on them make a move with a tool on the public chess board for the favor of the disabled mind’s in our world.

First I must admit that a disabled mind can be a result of many things but for many patients a disabled mind is a temporary situation compared to these who have physical disability and which for them I and many others do not have any way of helping mentally. For these I cannot help I am devoting a minute every morning concentrating my body BTU and embedding it with pure good using some old “chants” that my ancestors left for me as a tool which worked for them when they wanted to help others but couldn’t do a thing for them because of the distance or the limitations of reality.

So for these which I can help I wrote a filtering DB which in it’s raw form has only three states of definition “white”, “black”, “other”. Using this DB we can categorize url’s on the Internet as “looks good” or “looks bad” or “unknown” which will mean there is no verdict about the content yet.

This DB is a very robust and delicate tool that was designed to eventually be a weight based rating DB(black=128, 0=white) but it takes time to accomplish this goal(I have funds but more are welcome) so first something to start with and later a more perfected product to come.

How SquidBocker is being used as a
“Therapy  Assistent Tool”?

Each user is assigned a unique SquidBlocker instance and an Internet Surfing session is being started with the patient. Depends on the domination of the patient on the Internet Surfing Art we start with either a basic guided session which will have a leading subject in mind. Then it will follow a non-guided surfing session. In the guided sessions we are teaching the patient how to categorize the site\url and these can be:

  • Text News
  • Acadedmic Articles
  • Music
  • YouTube Videos
  • And many more

Every session have a “summary” of what we had and what we know now and also what do we understand from now on based on the session. It’s a journey into the depth of the human\patient soul in an interactive way using the WWW. This is not the right place to add pornography to the picture since it’s one of the most ridicules forms of content in the Internet compared to many others but it’s good to know that some do not understand what pornography is good for and what it’s bad for. Using a SquidBlocker categorizing session we can achieve one of the most amazing achievements in a therapy process.

All The Bests,
Eliezer Croitoru

References:

squid-3.5, squid-4.0, squid-release

Encryption, how far will you go? – SQUID 3.5.25 + 4.0.19 RELEASED

Encryption, how far will you go?
SQUID 3.5.25 + 4.0.19 RELEASED

Choosing the right path with encryption is a crucial manner in the modern Computers World but I want to take you far behind this era into 2 or 3 hundred years ago.

Back in the 17 centaury there weren’t as many options to encrypt a message, if you wanted to implement a 512 bits based encryption you would first need to invest a lot of money on traveling and security of the first public key exchanging leaving aside the trust you need to have for the key exchange location.
Yes it was a complex and an expensive task that only some had the luxury to have, but today we hear “Let’s encrypt” on a daily basis.
So I took couple minutes to try and understand how would a single mind be affected if it would be required to encrypt a conversation.

It’s not a normal thing to glance at, two persons sitting in a room and talking in an encrypted language. But the only thing they want to say to each other is “This is not a secure place, we need to go elsewhere”.

For this single phrase they had a full 2 hours session!
The first time I have seen such a thing I was stunned but then I started to delve into the depth of “The Encryption Hell”. It’s a place that is reserved for these who would like to talk using encryption on anything.

You must first understand that there is a lot of noise surrounding the plain text so you would need to have some way to handle the noise and also to handle the encryption and only then handle the actual conversation. This is the place of uncertainty! In such a world you cannot have some rest for your mind. The brain is always working and thinking about the possibilities. “Does this word means that I need to leave the room this second?” it’s really a place that is reserved for these who live in a deception on every move they do.

Here comes in handy the arguments about using open-source or closed-source based encryption systems. When you have an open-souce system you can share with others the methods the concepts and the ideas. Then you may have a chance to get into a better place then to stay in a “Stand Alone Complex”. But the closed source has to offer a lot!!

It is possible to have a closed source and still have the benefits of the open-source world just by being connected to the right sponsor. The myth that the closed source world cannot offer alternatives or advantages was proven long ago to be a false positive. There is no need to present an argument because the world works this way, you can have both security and secrecy!

An example for that is one of the projects I helped long ago to handle as a starter proxy servers admin. I was asked to give help with intercepted traffic analysis. Indeed I could write the proxy but there was some hash embedded into each message that resulted in the team trying to crack it clueless to what behind it.
The target was to falsify a 128 bit transmission that is being sent over a tcp socket from the client to the passive server. The hacks were required to hack only 128 bits. The end result was that after about couple days they said that the encryption is impenetrable!

What was the blocker, the lack of CPU? it was a team of more then the most talented hackers.
Was it the money? They got a lot of money only for the trial.
Was it RAM? they had a full fledged servers flat available only for their use.
So what was missing?
Two keys: The main private and the Diffie Helman one.
This technique has been in use in the world for hundreds of years but not many used it. Not many knew how to use it and not many where able to operate such a cipher system. This is one thing that took empires up and down: Encryption.

For us this “function” was off-loaded from the human mind into assisting disposable devices such as a SmartPhone while many do not even know or understand what’s under the hood. We can walk with an encryption device in our pocket without using any of our brain “CPU” and without touching our pocket.

So 128 bits aren’t that much but if you use them right they would be sufficient and you won’t be required to recalculate every second the escape route from you current position to a safe state or place.

One of the most used concepts is blending into big crowd to loose tracers. It’s not the most efficient way to do so but if you begin a session in a very noisy place there is a chance you would be able to exchange keys without anyone knowing. So today we have the option to get some level of secrecy without paying too much like in the old days.

Indeed for a money transfer you would need a big and well shielded truck but with a group of assassins or a well trained ex-army experts you would need couple tiny cars , lot’s of cash\gold\goods and a route. It is possible to secure a transfer without using the “Heavy Gear” but with the “Right Team”!

It’s a known way to run security and my way of things is to help others with it.

I asked myself couple times in the past: Should I start an encrypted session? And then I noticed that it’s not required to invent keys, I already have them. My ancestors left me with many keys and many ways to encrypt even in the harshest conditions.

I have a library which I use as a delta and reference daily to analyse and decrypt the most hardened and complex minds in the world. I can say that I have secrets but I daily lock them and throw the key. After each time I am throwing the key I am chanting a secret spell that was passed in my family for ages and I remeber that we are all in the same boat and in the moment I will try to dig a hole in our unified boat we will all sink together.

So how far will you go deep into hell to throw your keys? will you use your Cerberus to defend this ship? will you throw your given keys to hell ie the most secured place in the entire universe?

Diffie Helman is one of the most proven method for forward secrecy of encryption and I am using it daily long before it was introduced to the public SSL world. I took couple trips to hell because of it but eventually even the most notorious psychiatrists and therapists declared that I am a proof of a truely hardened human.

In my line of work as a Linux SysAdmin you must be Hardened! and you must know what the Linux “Talisman” is made of. You cannot blindly use it as is!!!

Long ago after the Squid-Cache RPM was downloaded more then 10k times I stopped counting. This is  since there is a hidden secret inside each and every one of these downloads “I trust you Eliezer Croitoru to provide me a true\good binary of Squid-Cache”.

Squid-Cache is a production ready product but it requires a very Hardened character to use it

Encryption is a challenge, if you are up to it jump into SSL-BUMP and see how and if you manage to make it work.
Throw your fear from a Server Crash!!! the Squid-Cache team is working hard so you would be able to test their ability to make you happy enough to see the magic that they can do. Indeed it’s not the perfect product but it’s worth just trying to see and understadnd what it might lacks.

All The Bests,
Eliezer Croitoru

References:

squid-3.5, squid-4.0, squid-release

The Internet as a Talisman – SQUID 3.5.24 + 4.0.18 Released

The Internet as a Talisman
SQUID 3.5.24 + 4.0.18 Released

Some have more and others have less meaning for things in their lives and specifically for objects and to objectives. Most of the kids I have seen in my life have something embedded into them but not every eye can see the same things. Depends on the background and nature of the person he or she can see beyond the flesh and blood.
There is some part of it in the form of genetic material but I and many others believe it’s not the only thing. Every kid has it’s own embedded and unreplaceable soul. We have the option to show some reflection of a fraction from our soul to others either by plain text or by some Talisman, there is meaning in things. Even the most notorious researchers cannot deny that we all have some “meta” things embedded into us which the genome cannot touch. In a similar way to programming languages we can operate on the lower or the higher levels of this “meta” world.

“What will I choose to show for all in my piece of heaven?”
Is a question which everybody should ask themselves to my opinion.
Some decide to use their own body to show things in a form of a tattoo but it’s a much inferior form compared to many others. The basic level is with what exists already like the food you can serve: junk or nutritious. For some there is a conflict between these two choices but it’s a fact you can serve both non-junk and nutritious. A demonstration of what is considered honesty might not look the same as a shiny tattoo but it has much more warmth embedded into it. It will also affect much more than a single image or picture.

Today we have all kinds of tools to demonstrate an idea but still all of these cannot be compared to a living creature. I cannot pin point the exact difference but it’s there.

The Internet can be used as one of your own Talismans in your tiny piece of heaven. The issue with this compared to many more static and defined things is that there is an illusion that the Internet is composed of static things. For example “google will always be there” was a saying a friend of mine told me. The issue is that my tiny monitoring daemon shows from time to time that google and couple of google systems were down for a while. So the Internet is not that static. It has some “endpoint” which some humans operates.

When we open this tiny Router box that connects us to the Internet we will probably won’t find the exact reason for some site to be down. But, what box will we choose to put there at the edge of the house? What box will we choose to represent the Internet in our Heaven? Will it be a piece of junk which barely pumps the bits or will we use a top notch one?

It’s a simple question which we need to consider. For most of the users that question is being relayed to the sales man and he makes some of the choice much simpler for us. But, for the more technical(techy) person the answer is much more complicated. The simple person will want content filtering and OpenDNS or Symantec DNS systems will be good enough. In the other hand for the technical person which knows what’s inside the box, in some level the choice would be much more complicated. And it cannot really be relayed to the average sales man. Then the question might arise: “Maybe I will put together my own self compiled router and filtering system?” some might be able to make it past some level of the task, but eventually compared to a team of developers, what level of depth of this task will a single person mind would be able to surpass?
And to make some sense into the question: Would a single mind be able to compose a Talisman strong enough to be able to surpass the power of so many others out there on the Internet? Will a single person mind be able to make it so Internet surfing\browsing be smarter then viruses, malware, porn and many other things which most sane persons will not want to be affected by?

Indeed, it all starts with education but it also stats with a single box which exists now in so many homes, a PC that is connected to the Internet through a router. And do not think that I have some grudge against Facebook or YouTube or China but as much as I want to tell little kids how great is the experience of a marriage bond I would still block them from irrelevant details of this bond. It’s a part of a wisdom that I was granted  me freely without any form of payment.

Discretion is a wisdom and every Talisman has it’s own “Real” value, and yet maybe the sales man will tell you stories about the benefits about the product, but, eventually what you decide to buy is either the story of one sales man or another. This is since you cannot disconnect yourself from the World or the Internet. We need others which resides on the other side of this World Wide Web.

I had the chance to sit with some of the top notch developers in the content filtering world and when you ask them about their creations these do not even scratch their ideal solution. But yet so many enjoy their work so much. For the end-user, which doesn’t understand the complexity of the task, the “simple” box that the developer didn’t liked much might be the best choice of all.

Squid-Cache is one of these creations which are open to the public and can get criticized but yet to be replaced by other solutions for years. But why? Because it’s free? Because it’s Open-Source?
The answer is very simple: The developers behind the code.

The developers are the Brand!!
Like a chef they work on every release to prepare the next recipe of a Talisman which contains lots of goodness embedded into it.

Will it work for you?
There is an option that it will not fit your needs!!!

So what should I do?
Should I choose Squid-Cache or another product?
The answer to this question can be answered by a trial and error. Like a date, we put our efforts and try to see  the good in the other side of the contact.
Choose your Talisman: a 80’s compared to a 21’th century
I can only say that if you are looking for a ready to use off the shelf product Squid is not for you!
(Free Hints for research, try: ClearOS, NethServer, SmoothWall, ZeroShell, Zentyal, Untangle, Endian, pfSense, OpenSense )

Squid-Cache is one of a kind, a HardCore part of the web!
It’s the Software which will open your mind to a whole new HTTP world you have yet to see in the past.

All The Bests,
Eliezer Croitoru

References:

squid-3.5, squid-4.0, squid-release

When is the kid considered stable? – Squid 3.5.23+4.0.17

When is the kid considered stable?
Or
When is the software stable enough?

I am not developing software daily but it seems that the inner debate on the stability of our different “kids” and fruits of work is always there. Every time when I consider something as stable enough for production some other voice adds a doubt on the stability of a software.

Specifically, in the squid-cache and the open-source world the question is always on the board or the desktop. The source is open to everyone to find the next bug. Some are happy with what they have already while some expect the equivalent of a Ferrari.

When I am writing a software, I am doing my best effort to write it with the main goal that human lives can be entrusted to this software. One of the reasons why I am trying to meet such a goal is that in real life I try to do the same. When someone asks me a question or turns to me with a request or a word, I know that he asks for a reason. There is no chance in the world that the occasion is a result of only “A series of of unfortunate events”. I was asked in my past couple times “Why do you bother to answer?” and the answer is the simplest: the sanity of the other party is in my hands.

Sometimes a REDIRECT is the right answer but never DROP or REJECT. These are the actions of “war” and when we are talking about HTTP in general there is a war out there.  I had the choice in the past to work in couple layers of the Internet from the hardware to the application and I choose to invest lots of time on layer 4 and above.

There are many tools in this warzone and every time that a new tool is in use it get’s it’s own life cycle. The issue is that it takes time for every tool to become mature enough to serve different purposes.

Squid 3.5 is already in use by many users and admins for a while and is considered Stable for a very long time, but now aging start showing up.  There are different levels of maturity but the basic one is a period of 30 days uptime. We do expect more but a restart once every 30 days without any crash would be considered stable.

For now I am in the hunt for fatal bugs on the 3.5 series. The reason for this is to measure the maturity of the the branch!

Lately I have been working on couple tools and one of them is a library for distributed rate blacklists querying and blocking. Using this library I wrote an external acl helper for squid that can help many admins to use OpenDNS and Symantech or other internal DNS Blacklists.

The library sources can be found at:
https://github.com/elico/drbl-peer

And in binaries package at:
http://moodle.ngtech.co.il/drbl-extacl/

squid.conf example of usage:

external_acl_type dnsbl_check ipv4 concurrency=200 ttl=15 %DST %SRC %METHOD /opt/bin/squid-external-acl-helper -peers-filename=/opt/bin/peersfile.txt
acl dnsbl_check_acl external dnsbl_check
deny_info http://ngtech.co.il/block_page/?url=%u&domain=%H dnsbl_check_acl

http_access deny dnsbl_check_acl

Example of peersfile.txt:
http://moodle.ngtech.co.il/drbl-extacl/peersfile.txt

The syntax of the above file is:
type<space>address<space>path(for http services)<space>port<space>rate for the host(uint)<space>address which will indicate a blacklisted domain with spaces between them.
The type options are:

  • dns
  • dnsrbl
  • http
  • https

The http and https destinatiosn are queried with a HEAD request to the host and path and a match will be reflected in the response headers with “X-Vote” value to be “BLOCK”.
The dns and dnsrbl will be a match for one of the addresses which are defined from the six value on the definition and on ie matches in the line:

dns 208.67.220.123 / 53 128 146.112.61.104 146.112.61.105 146.112.61.106

would be the addresses:
146.112.61.104 or 146.112.61.105 or 146.112.61.106

and the weight of a match is 128 and specifically the default match weight of the helper is 128 and this line would be a match and no more lookups will be done. In the case which one list is not a match each of the listed would be tested until either found or the timeout(default 30 seconds) is reached. If the timeout or the list of peers was not found as fully matching the weight, the request will be allowed.

With this tool you can use regular DNS services on your system and on the proxy intercept the traffic and get a decision using a “consultation” with an external system not compromising your clients with special block pages that the dns redirect towards.

So we have both a proxy and a simple tool which can help us to prevent access to specific sites. The stability of Squid for this release is considered “Very Stable” but yet to be tested on a larger scale then 400 users. If you are managing a system which runs squid for filtering or caching that have more than 400 users please send us some input from the squid manager info page so we would be able to rate the state of the software.

I am planning to write a tiny tool\script that will help to scrap the squid manager info page and send the Squid-Cache systems some feedback. If you are a squid system administrator which are willing to share some statistics on your system with the project, please contact me at: eliezer@ngtech.co.il

I believe that if we could gather enough statistics we would be able to declare that the software passed the “masses” test compared to couple single systems.

All The Bests,
Eliezer Croitoru

On the plate:

References:

squid-3.5, squid-4.0, squid-release

The Squid “Persona”- Squid 3.5.21+ 4.0.14

Who Is The Squid Girl Persona?

squid_girl__shinryaku__ika_musume__minimalism_by_greenmapple17-d8u9mum

As a Squidder I know a girl or two but others are in the lookup for their Squid Girl Persona. Its’s not a simple task!!
To illustrate the idea: We as males used to the web with all sort of GET, POST, OPTIONS, PROFIND and all sort of normal methods.
But the Squid Girls is something else. She has brains!

I mean what would an empty shell be like? Would a Squidder would ever want a SHELL only system? Won’t some graphic or UI would be nice?
We do need some humen with enough Persona to take the SHELL up. As a friend told me once: We have great code, great kernel great compiler but.. with only one missing feature! The UI can be customized.
What can you say to someone like that? I mean, if the Software give you a full list of OPTIONS for the UI, what to choose from?

Despite to me preferring SHELL and scripting languages I believe that a good ICON and a GOOD Persona will give good taste for the whole cup.

Now let say we have a full stack proxy(AKA Router) and we can connect a network of Squids, what would be the next step?
Would it be plausible to “share” a cache_dir?
What about a ReadOnly cache_dir?
It sounds a bit weird but with good Cache Operators I believe that it would be possible to enhance a Squid-Cache network to serve a global Sharing system(compared to a CDN).

Google, Facebook, Yahoo MS, and couple others can be the main INPUT systems and the Squid-Cache Operators can push content into the right network at the right place and the right time.

The Squid Girl persona would be there just waiting for the Squidder and using a proxy, two, three, a BIG network two Squidders can be connected.

I am happy to say that from mu side of the picture Squid 4 feels stable to me and others needs to confirm that it work as expected for much larger systems.

Eliezer Croitoru

On the plate:

  • Squid full Tproxy LoadBalancer with The proxy protocol connection initiation.

Refrences: