I just did some testing and it's even worse than that: 7-zip completely ignores the file extension and snoops the file format based on the file contents only.
So simply opening a malicious ".zip" file on Windows could trigger the HFS+ vulnerability. Using 7-zip to open any file means you have a HUGE attack surface.
EDIT: One simple way to mitigate this issue would be to just throw a confirmation prompt if the file extension matches a known format but 7-zip is about to run a decoder for a different format.
Not on Windows, at least. The properties dialog can tell you it uses HFS+ if you ask for it, but it's too late because the HFS+ code has already been executed.
I am appalled at how unseriously 7-zip seems to take security.
The changelog only says "Some bugs were fixed", with no mention that there are serious security flaws. The homepage doesn't mention any vulnerability.
The installer is not signed, downloads are over HTTP only, and there is no hash available neither on the homepage or on the forum announcement linked from the homepage (the latter is served over HTTPS so it would be a reasonable option); thankfully you can dig around Sourceforge downloads to find a SHA-1 automatically generated by Sourceforge. So I can personally verify the integrity of downloads, assuming Sourceforge can still be trusted, but 99% of users won't do this, and more worryingly it's a strong signal that the developers may not understand or value security very much: you have to wonder if maybe they're not themselves downloading unsigned software over HTTP all the time.
7-zips track record of lack of vulnerabilities [1] would imply something entirely different to me. Only 2 CVEs ( and now this ) in a heck of a long time. 7z being used in quite a variety of places it seems like a lucrative attack vector, so I would expect that ample amount of fuzzing and other techniques has gone into trying to break it over the years. Which obviously doesn't mean, as these findings highlight, that it would be in any way infallible either. But to say they don't take security seriously sounds little unfair to me.
Yes, the number of reported vulnerabilities isn't a good metric to judge the security of a project. It often means that few people bothered to look, and that when developers fixed a bug they didn't try to find out whether it was exploitable.
The latter is supported by the fact that changelog entries don't discriminate between security bugs and normal bugs, and by the fact that no vulnerability was ever reported by the developers themselves.
More worryingly, in all instances where a vulnerability was reported, the CVE vulnerability was filed specifically because the reporters were security researchers. This means that when a normal user reported a crash bug, no vulnerability was EVER filed. How likely do you think it is that none of those crashes could possibly be turned into an exploit? There are 95 instances of "Some bugs were fixed" in the changelog.
Suppose researchers looking at 7-zip were lazier getting their CVEs than those looking at Winrar, Winzip, Info-zip, zlib, bzip2, and other commonly used compression packages. ;-)
You can be appalled, but it's not like anyone's paying them for the work that they do.
If you want to help them, I'm sure you could contribute some additions to their build process or something that would help them tighten up security. But I think it's funny that you're so shocked that a popular free software project isn't perfect.
Be the change you want to see.
PS: who cares if the devs are using unsigned software downloaded over HTTP? I care about using signed software (and then I suppose the transport doesn't really matter), but that's totally unrelated to what the devs do on their own computers.
Every time an open source project is criticised, you get people saying "why are you criticising, you should do the work" - to the point that this seems to be a mechanism to shutdown any and all criticism of any open source project via special pleading.
One can be very grateful for the work done on an open source project, and recognise that I have no right whatsoever to expect them to hop to it, but I am always free to criticise their work whether it is free or not. Just because a project is open source doesn't mean nobody can utter a bad word about it.
Thank you. To be clear, I don't mean that as a criticism of the developers, who as the parent points out do very useful work and do it for free. But I feel that it's important to have an objective look at the current shortcomings in the state of 7-zip security, both in order to understand what needs to be done to fix it, and in order to warn current users until those issues are fixed.
7-zip is a widely popular basic utility, like a web browser. A flaw in 7-zip is very serious, because as I pointed out elsewhere simply opening a .zip file will allow an attacker to exploit it. And while there is a strong security culture among web browser developers, 7-zip doesn't seem to have that culture (yet).
There is certainly a massive budget and manpower difference, but a lack of mention of security fixes in the changelog and a lack of hashes isn't a manpower issue, it's a culture issue.
As a side note, compromise of a developer's machine is a big deal in my opinion: it could be easy for a criminal entity to slip in a tiny change in a large patch that introduces a vulnerability; and depending on how builds are performed, a criminal could patch the final .exe with no visible change to the source code. These are tailored attacks, but for a very widely distributed program it would easily be worth the criminal's time.
I'm glad you care so much, but I don't think you can fix a culture issue by explaining it away. The best way to set a culture where there hasn't been one before, is to lead by example.
Re: side note; the vulnerability described in the well-known Ken Thompson paper has been exploited just once in the wild. It's cool, but you could say the same thing about trusting Windows or proprietary drivers or hardware.
I counter the Thompson claim as vastly overstated risk when I see it here. I hadn't even heard that it was ever done before. Do you have a link or the project/time? I try to track these things.
Thanks for digging. I'll be damned! Somebody did pull it off. On my old, favorite platform as well!? So, one time on record.
Still supports my claim that reproducible builds and Thompson are mainstream buzzwords where our real concern per Orange Book days should be: coding defects in compiler source; effects of optimizations; malicious developers; trusted distribution of source; bootstrapping first, verified, local compiler. That's basically a human and machine verified compiler with simple code and signed zip's. Knocks out Thompson attack as side effect and negates reptoducible build need except for debugging.
I'm not talking about a "trusting trust" attack, which is difficult to pull off and requires special compiler knowledge because it needs to survive bootstrapping.
Here the attacker just needs to patch a binary once and he already has complete control over the machine, so he has an infinite number of options: from simply manually replacing the binary file before it's uploaded to the website, to replacing gcc with a script that patches the source code before calling the original gcc.
It's not shutting down the criticism. Someone will always say that ("you can fix it blah blah blah") but you can just ignore it. Just don't expect anyone to put in extra hours fixing it to avoid that criticism because people who do open source work for fun in their spare time, and people who give a shit what random people they don't know say about them online are two entirely separate groups.
Anyone's within their right to criticize an open source project. I think being "appalled" is a tad illogical, though.
Similarly, does complaining about it on a public forum have any effect? In my book, code speaks louder than words; if you really want to see change, you know what to do.
Under many circumstances, our actions come with a certain degree of responsibility. When we write code, we have the responsibility to be pretty open about security.
Where's the users' responsibility factor in to support the project financially or with code contributions? A gift given as is implies no responsibilities. A tool given for money should work as advertised. Yet, all this talk of responsibility appears on FOSS projects and is one way.
They have no responsibility to anyone. If you want responsibility, there's commercial offerings or you can sponsor 7-zip security for money.
Our actions do come with a certain degree of responsibility. It is the user's responsibility, then, not to use software that is "without warranty" if they do not firmly trust it.
You can claim irresponsibility whenever you want, but it takes two to tango.
Not everyone has enough bandwidth in their lives to actively fix every broken free/open source project out there.
Rather than being seen as attacks on a project, these sorts of comments are often just highlighting some things that whoever is looking at that particular project might want to fix.
The "don't criticise unless you're prepared to fix it" attitude is very similar to the sort of "Don't bring me problems, just solutions" attitude that some managers have that results in a culture of people keeping quiet about stuff they've spotted but don't have either the time or ability to fix themselves.
I made specific criticisms about the parent poster being "appalled". Those are the kinds of comments that programmers on open source projects don't read, because they start "I'm appalled at how irresponsible your project is", and no one wants to read something like that.
Rather than leaving comments about how astounded we are, we should come up with a solution (which I'll note wasn't even mentioned in the parent post). Maybe a build step to autoregen the Downloads page? Maybe an email to the developers that will ask nicely for them to note which bug fixes are for security and which ones aren't?
Omission of facts is sometimes a defense mechanism for people who are embarrassed. The parent post could have only made the developer's insecurity about security greater. I understand that we have to make criticisms about open source software, but maintaining an open source project can sometimes be totally thankless work. We should try our best to Be Nice to people who work for free, because otherwise they won't want to work at all.
> PS: who cares if the devs are using unsigned software downloaded over HTTP? I care about using signed software (and then I suppose the transport doesn't really matter), but that's totally unrelated to what the devs do on their own computers.
This is definitely a vector that attackers can and do use. If the developer is infected, particularly by a virus that changes the compiler to emit infected code, this can by proxy infect the products they develop.
Can you give me pointers about authenticode flaws? After a quick search the two issues I found were with MD5 collisions (solution: the signer should never emit MD5 certificates) and certificate padding (solution: the user should set a registry key to enable verification of padding; search for "EnableCertPaddingCheck").
Their conclusion is simply that "the major part has been fixed" in MS12-024, and that you should be careful if you write a self-extracting installer. No big deal.
All of them are self-extracting installers (happen to be 7zip/Nullsoft). That successfully fake sigs. Nullsoft is the most popular packer/extractor out there.
Sure, there are ways of creating an installer with authenticode that cannot be faked, but much easier to just hash it and not worry about the terrible tech that is authenticode.
Edit: That's a fake Firefox installer with authenticode that checks out according to the spec. As you can see, this is not some weird edge case.
The SHA256 of that file is exactly that of Firefox Setup Stub 35.0.1 (Win32), so of course Authenticode checks out.
I'm not 100% sure why some people thought it was a malicious file, but the comments on Virustotal mention [1] which, for me, redirects to the legitimate [2] but a comment seems to link it with [3] which is a completely different file. Perhaps the redirection is randomized so people got confused?
[1] hxxp://files.dodo-number-1.pw/p/MCLkP8Dzc3nUWJrG9fwGLA,1442015273/zte%20mf631%20firmware%20downlo_10924_i57945825_il345.exe (replace hxxp with http)
There is some very shady SEO going on here. The French translation link goes to a Wordpress blog with a very bad translation of the introduction. The root of the translation website is a completely unrelated Dutch used car website.
Other pages on the translation blog follow the same pattern: an old webpage (this HN entry is from 2009, but other pages go back to 2000) is edited years later to link back to a bad translation in some random language.
This is very far from my area of expertise, so I'm curious to know if this is well-known, and if those pages were hacked or if this was inserted by the owner himself.
EDIT: This is fascinating. I'm finding dozens of similar blogs, with links spanning at least a thousand hosts, which I'm assuming were hacked. This is a big network, with sophisticated wording designed to look as innocent as possible.
The content seems to be written by a hobbyist decompiler developer. Unfortunately, the state of the art that it presents seems to rely heavily on the Program Transformation Wiki pages, which appear to have become rather defunct (its pages, on further research, appears to have been primarily maintained by Mike Van Emmerik, and both Van Emmerik and Cristina Cifuentes appears to have moved on from binary decompilation a while ago).
As a result, the academic state of the art that it's relying on is a decade out of date. While decompilation is in nowhere near a solved problem, there have been advances in variable and type recovery in academic literature that the author of these pages doesn't seem to be aware exists.
The content of the pages mostly seems to be the sort of basic material that I wouldn't even bother citing in a research paper. I'm not sure there's any information in there that you wouldn't find in, say, the dragon book.
I just downloaded the dataset, and color is such a powerful feature that training a random forest on images downsampled to a single pixel results in 95% and 98% accuracies! (for the 4-category and 6-category versions, respectively)
And you can easily exceed 99.5% by adding more features to the forest, which is far above their DBN accuracy.
I have no idea how they were able to get an accuracy as low as 69% when they evaluated random forests.
I read the paper, and I also have some reservations. The procedure they used to extract and randomize their data seems biased towards large homogenous areas.
In short, in their procedure, it seems possible to rope off a large contiguous area of Mojave desert, ground-truth it using their GUI system as "barren", and have that area be carved up into 28x28 pixel chips and spread equally into the training and test sets.
In such a case, the training and test sets are not really independent. And their 6 classes, as you point out, are amenable to color features.
Having done classification of remote sensing data...the above is not a good test of accuracy at any useful task. You have to test accuracy on representative data.
That means training within a few areas, and testing on geographically distant but ecologically similar areas. (I.e., same class, but statistically independent.). And, varying things like time of day, observing geometry, and seasonality. Color features will be quite fragile in such tests.
And, testing on a more diverse sample, to see if "none of the above" can be detected, because their class decomposition is nothing like exhaustive.
> the one other researchers (which I respect much more) discovered in 2006
You seem to be blissfully unaware that they're in fact the same researchers: André Seznec is a co-author of the paper.
This paper is valuable in pointing out that ITTAGE branch prediction performance is a very good predictor of Haswell performance. Because the Haswell algorithm is secret, that should be very helpful to developers who still have to care about branch prediction.
Interpreter developers shouldn't ignore pre-Haswell chips just yet, but if you were to, say, develop a new language your design decisions should be guided by where the puck will be rather than where it used to be.
> You seem to be blissfully unaware that they're in fact the same researchers: André Seznec is a co-author of the paper.
He's the third signer of the current paper but the first of the 2006 one. It's the case of legitimizing the "research" which only confirms exactly what Intel recently implemented in a given CPU generation, even if the grant is for other goals ("ground-breaking, high-risk projects"). Or, to be clearer, I respect the deeds not the persons.
I fully understand the need for Seznec to be able to claim that the algorithm is really used by Intel and that therefore he co-authored the last paper. But confirming that Intel used ITTAGE is "ground-breaking, high-risk"? No. And does that mean we should never use goto label in the tight loops? No.
The grant is for the DAL project [1], which presumably fits the "ground-breaking, high-risk" label. In such a project, some tasks will be high-risk, some tasks will be low-risk. Quantifying the branch predictor of current processors may be a comparatively easy task, but that doesn't mean it's trivial, useless or outside the scope of the project, which is to improve sequential performance of microarchitectures. Knowing that real-world processors are performing just as well as previous academic research is helpful, because it suggests that there is no difficulty or unrealistic assumption that prevented manufacturers from doing so. Conveying that knowledge to the compiler and interpreter community is important too.
There are no absolute values in the summands, so in your example pairs of consecutive terms sum to 0 when you choose b = 2.
The theorem seems entirely correct to me. You can prove it with these sub-steps:
(1) the set of all j + a_j is the set of nonnegative integers minus a finite number of gaps
(2) thus for large enough n you can express \sum_{j=1}^n (j + a_j) as a quadratic function of n, plus a residual term e(n) of magnitude at most 1007^2/2
(3) more precisely, \sum_{j=m+1}^n a_j = g (n-m) + e(n) - e(m) where g is the number of gaps in (1)
What the article doesn't state clearly is that this assumes the strong exponential time hypothesis: it assumes that SAT cannot be solved in time 1.9999^n -- in other words that it's impossible to do better than the brute-force algorithm, which has complexity 2^n (up to polynomial factors). That's a very strong assumption.
Just 2 weeks ago, I won a machine learning contest with a $20,000 prize pool where the goal was to predict the IQ of a child at age 7, based on various biological measurements and demographic indicators. The data includes whether a child was breastfed.
After reading the article I did some very quick computations and my finding based on the model I developed is that children who were not breastfed have an IQ impairment in the 1-3 IQ points range, after accounting for confounding factors.
This is very consistent with the results published here.
Compared to this study, I believe the methodology I'm using is more powerful for three reasons:
- much larger sample size: the dataset I have access to comprises 12015 children, compared to 3493 for the study
- a larger set of confounding factors is accounted for: notably, the data includes height and weight measurements at up to five points in time
- confounding factors are fully accounted for, rather than hand-waved away. This is a complex model based on random forests and linear model, and the results are entirely cross-validated.
Stay tuned for more detailed computations. I will also ask the organizers for the exact definition of breastfeeding used.
---
Edit:
On the other hand, the study is still very appealing because according to the authors, there is little correlation between demographics and breastfeeding in Brazil, whereas the validity of the effect I'm reporting is dependent on whether the demographic model is powerful enough to remove the correlation. Still, I believe most of the correlation is easy to remove, and it isn't clear that there aren't subtle demographic effects even in Brazil. In particular, the proportion of participants with missing IQ data seems to decrease with duration of breastfeeding, and I don't know if they have an explanation for that.
I compared my dataset and their data, and it turns out that even in Brazil there is significant correlation between mother education and breastfeeding, barely less so than on the dataset I have. So you should probably disregard my edit in the above post: accounting for confounders could be important in both cases.
Thank you for taking the time to comment about your work.
I came to comment on the headline, basically to say that "Breastfeeding 'linked to higher IQ'" sounds awfully backwards -- surely this isn't really a result "in favour of" breastfeeding, but rather pretty damning evidence against formulas/substitutes ?
I like your formulation much better:
> children who were not breastfed have an IQ impairment
I see the article climbing back up the front page, and I want to make it abundantly clear that I don't believe that there is necessarily a causal relation from breastfeeding vs formula to IQ.
I could control for demographic factors, but not for other major important factors in a child's development -- parental IQ or views regarding parenting are huge factors in a child's IQ, and they may also influence the decision of whether or not to breastfeed. If parenting books unanimously decided that formula was bad, then parents who cared enough about their children to read them and follow their advice would be more likely to breastfeed and you would see a positive correlation with IQ, even if formula was completely equivalent to breastfeeding.
So while there is a clear correlation after controlling for demographic, there is not necessarily a causation. My formulation really wasn't meant to imply causation, but rather the handling of unknown data (I grouped unknown-status children together with breastfed children). When talking about correlation, it's not meaningful to make a distinction between IQ impairment and IQ gain.
The contest was hosted on TopCoder. The name of the organization who provided the data and funded it is not public, but I'm told it will be made public at some point.
I've never gotten my IQ tested, but when I read about studies like these - they usually demoralize me from fear of competitive behavior. I think this leads to low self esteem, and the imagining that I have a low IQ (high marks through schooling and post graduate education, MSc, possible imposter syndrome) as my default attitude. It can be a self defeating attitude, but what it usually does is make me argue with people about IQ tests, and explain why they are not necessarily indicative of intelligence, individually or globally.
I can imagine that merely the act of measuring IQ has significant effect on the population. I can not imagine a population that exists without it, but I imagine it would also have a huge effect on society, as you similarly hypothesize about the 1-3 point increase across the population globally.
This, but also, IQ isn't an exact score, right? Wouldn't this fit snugly within the margin of error?
Not only that, but are people even still using IQ as a metric of intelligence? As I understand it IQ is only a measure of one type of intelligence. I mean I'm all for giving a child every advantage you can fathom as a parent, so I think this is still kind of cool research, but I don't see how it equates to anything truly meaningful.
It kind of just adds one more point in the "breastfeeding is good" column.
I found a big improvement to my model just after the contest was over, so I made a proposal to develop the improved model and do some more in-depth analysis. I'm currently waiting for their reply!
If they accept I'll have an opportunity to look deeper -- it's one thing to develop an efficient model, but fully exploiting it in order to gain a better understanding of the data takes some work. A limitation of these contests is that you're rewarded for producing a very efficient model, but there is very little emphasis on analysis of your model once you built it. I think it's a shame, because the person who built the model is often in the best position to have a good intuition of both the dataset and why the model had to be built that way.
I've been considering opening a blog, but I haven't found time to do so yet.
Briefly, the purpose of the contest wasn't to understand the effect of breastfeeding, but to understand how important normal child growth is to mental development. They included several scenarios: with all data available, with demographic data removed, and with demographic data and growth curves removed. Unfortunately, IQ is so overwhemingly affected by demographic that the scenarios without demographic data devolved into a game of extracting all the demographic data that was leaked by non-demographic variables. And when demographic data is available, more than 90% of the variance extracted by the model comes from demographic data rather than biological measurements!
It's really disheartening to think that depending on the social setting you come from, you start with an IQ of 85 or 115 -- at age 7...
So simply opening a malicious ".zip" file on Windows could trigger the HFS+ vulnerability. Using 7-zip to open any file means you have a HUGE attack surface.
EDIT: One simple way to mitigate this issue would be to just throw a confirmation prompt if the file extension matches a known format but 7-zip is about to run a decoder for a different format.