Quotas for Ephemeral Storage

This proposal applies to the use of quotas for ephemeral-storage metrics gathering. Use of quotas for ephemeral-storage limit enforcement is a non-goal, but as the architecture and code will be very similar, there are comments interspersed related to enforcement. These comments will be italicized. Local storage capacity isolation, aka ephemeral-storage, was introduced into Kubernetes via <https://github.com/kubernetes/features/issues/361>. It provides support for capacity isolation of shared storage between pods, such that a pod can be limited in its consumption of shared resources and can be evicted if its consumption of shared storage exceeds that limit. The limits and requests for shared ephemeral-storage are similar to those for memory and CPU consumption. The current mechanism relies on periodically walking each ephemeral volume (emptydir, logdir, or container writable layer) and summing the space consumption. This method is slow, can be fooled, and has high latency (i. e. a pod could consume a lot of storage prior to the kubelet being aware of its overage and terminating it). The mechanism proposed here utilizes filesystem project quotas to provide monitoring of resource consumption _and optionally enforcement of limits._ Project quotas, initially in XFS and more recently ported to ext4fs, offer a kernel-based means of monitoring and restricting filesystem consumption that can be applied to one or more directories. A prototype is in progress; see <https://github.com/kubernetes/kubernetes/pull/66928>. ### Project Quotas Project quotas are a form of filesystem quota that apply to arbitrary groups of files, as opposed to file user or group ownership. They were first implemented in XFS, as described here: <http://xfs.org/docs/xfsdocs-xml-dev/XFS_User_Guide/tmp/en-US/html/xfs-quotas.html>. Project quotas for ext4fs were proposed in late 2014 and added to the Linux kernel in early 2016, with commit