[lttng-dev] [RFC v2] Add kmalloc failover to vmalloc

[Mathieu Desnoyers]

----- On Sep 21, 2017, at 4:30 PM, Michael Jeanson mjeanson at efficios.com wrote:

> On 2017-09-21 14:52, Mathieu Desnoyers wrote:
>> ----- On Sep 21, 2017, at 2:43 PM, Michael Jeanson mjeanson at efficios.com wrote:
>> 
>>> On 2017-09-21 14:39, Mathieu Desnoyers wrote:
>>>> ----- On Sep 21, 2017, at 2:29 PM, Michael Jeanson mjeanson at efficios.com wrote:
>>>>
>>>>> This patch is based on the kvmalloc helpers introduced in kernel 4.12.
>>>>>
>>>>> It will gracefully failover memory allocations of more than one page to
>>>>> vmalloc for systems under high memory pressure or fragmentation.
>>>>>
>>>>> I only used it in lttng-events.c as a POC, most allocations fit into a
>>>>> single page so I'm not sure how useful this actually is.
>>>>>
>>>>> Thoughts?
>>>>
>>>> Just update the changelog to specify why we need this for struct lttng_session
>>>> (~32kB), and you can submit it for good with a "Fix: " tag.
>>>
>>> I'm not sure it should be a fix, for this to be of any use all the other
>>> allocations bigger than PAGE_SIZE should also use it. Otherwise, the
>>> session allocation will succeed but the next sizeable allocation will
>>> still fail.
>> 
>> What other allocations larger than PAGE_SIZE using kmalloc did you spot ?
> 
> This is not an exhaustive list :
> 
> lttng-context.c:
> - new_fields: 136 * n
> 
> lttng-context-perf-counters.c:
> - events: if you have more than 512 CPUs
> 
> lib/prio_heap/lttng_prio_heap.c:
> - new_ptrs
> 
> 
> But mostly in the ringbuffer, thought I have no idea if a failover to
> vmalloc is acceptable in this case.

As long as we issue vmalloc_sync_all after vmalloc, yes, it's acceptable.

Please provide a patch that changes all allocations that can be larger than
PAGE_SIZE to the new scheme.

Thanks,

Mathieu

> 
>> 
>> Thanks,
>> 
>> Mathieu
>> 
>> 
>>>
>>>>
>>>> Thanks,
>>>>
>>>> Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com